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ABSTRACT 

This bulletin bresenis four papers on how to control, 
manipulate, and predict the rekdabi-lity of printed materials. The 
first paper describes trends in readability brought about by research 
tools developed by psychologists and linguists. The second paper 
explores the effects of word' frequency in printed materials on 
comprehension and concludes that we tend to recognize and understand 
readily the moire frequently used words. Experimental studies which 
contribute to the improvement of "the readability and teaching , 
capacity of preschool books are discussed in the third paper which 
also illustra^ies how readable preschool books may be created by 
exploiting children's love for television and cartoons. The fourth 
paper/fe an' evaluation of the use of the cloze procedure to determine 
the ^itability of reading materials, 
brief summary anci a bibliography, 
published in the 
(NS) 



Each article is followed by 

, ^ 3 These articles wer^. originally 

196"7 and 1968 issues of "Elementary ^English. " 
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Intrbduction 



Tlu» p«ipi*rs cuiilaiiu'tl in this biilK'tin 
acKlrovs llit-mselvi's tu tlit* iiucslioii of what 
makes llir langiiagf in inaltrials t»asv or 
(lilficull to riMcl. This (jnc\stion is of ti ntral 
uuportancr in thi* inhKiitionijl process, for 
, much of what a student learns he ac(iuires 
through his stud^ of written instruetional 
in«i^erials. The rese»»rch Uhich concentr»ites 
on finding the answers to this question has 
trachtionally been e»dled re»ul»d)intv re- 
search. 

The ehief aim of earl\ re»Kli»l)iUt> re- 
search was to devise formulas which edu- 
eators eonld use to determine if mateii»ds 
wiTe suitable to, their students. Because 
psyehologual and hiiguistie theories were 
as vet poorl\ (h veU)ped, the objectives of 
the early researchers were, necessarily 
limited and pragmatic. ThatMs, they merely 
soui;lit to de\ise an easiK applied formula 
which would predict the reading ease of 
materials. They could make little headway 
in atteinpiing to find the sources tliat 
caused the dilliculty. 

Modern readability ri»searchers view 
their task more broadly. They seek to 
establish. seiendfieally the principles which 
will p(Minit us not only to predict the 
dilliculty of niaterials l)ut also to write the 
materials to have any level of ililficulty 
we choose. Stated another wav, instead of 
just trviug to find coirelations l)elwee^i the 
features of language and its difliculty. 
modern readability reseaichers seek to 
analv/e the psychological piocesses in- 
\olveil in language comprehension and 
composition so that they can uiidtTstand 
why those* correlations exist. This is not to 
say, howmer, that rctulabihty resraichcts 
ha\e abandoned the use of (onekitional 
research teeliiiiqu(»s. To do so at this stage 
in the development of our knowledge of 
uMdabilily. would be foll\, There are 
literally thousands of featiiies of language, 
any number of which might be an im- 
portant factor in reading ease. As yet, 
ps\cholinguislic (lieoiy is too crude to 
O ritiit the (I priori exclusion of aiiv of 
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these variables as an important factor. 
Until such a llicory has been constructed, 
rcse«iiclicrs iiiust simultaneously consider 
d large number of variables, Correhrtioiial 
techniques piovidc the best available 
method for doing so. 

The point that should be emphasized is - 
tlhit, wiiilc modern readability researchers 
letain some of the techniques used by 
earlier researchers, they have abandoned 
much of the purely pragmatic approach 
which pervaded much of the early research. , 
The ultimate goal of modern corrclatiowal 
icsearch is to construct a theory of reada- 
bility which will permit us to reduce this 
arc»i of research to an experimental science, 
for to be sound a ' technology of written 
instruction can lie based only on the 
findings of experiments. 

Education stands to gain much from the 
fact that readability researchers have de- 
finesl thcii, task more broadly. Out of 
modern readability research is grovvhig a 
biKly of scientific knowledge about the 
iKiture of language compre|ieiision and 
Kmguage composition. This knowledge 
pioviding the basis out of which a tech- 
nology of written instructional materials 
is growing. This knowledge is providing 
the basis of a theory of the psychological 
nature of the processes people wtili/.e in 
the comprehension and composition of 
language. With this knowledge it will be 
possible to make substaiitiul iniproveiiieiits 
in the curriculum and instruction in reading 
coiiiprelieiision and language composition. 

The papers contained in this series re- 
flect this newer approach to readability. 
Their authors analyze their subject matters 
in much greater detail, base their analyses 
upon tiie general bodies of linguistic and 
psychological theory, and address lliem- 
srlves to the (piestion of how readability 
niav be deliberately controlled and manip- 
ulated as well as to how it may be pre- 
dicted. 

John H. Borniuth 
Editor 
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Teachers of reading and language arts 
are avid students of readability research. 
The objective of readability research is to 
find out what features of language influence 
the difiiculty children have in learning the 
kaowledge expressed in language. The re- 
sults provide the specialists with the in- 
formation they need to tailor instructional 
materials to fit the reading abilities of 
their students and with readability for- 
mulas by which they can determine if 
materials already prepared are suitable for 
their students. Fiiiajly, by studying how 
the many features of language influence 
comprehension, readability research pro- 
vides insights into the nature of the com- 
prehension process, itself. , , 

Recent Advances. The past few years 
have seen rapid and somewhat startling 
developments in readability" research. For 
example, the readability formulas available 
only three years ago could, at best, predict 
only 25 to 50 percent of the variation we 
observe iti the difficulties of instructional 
materials. Today, we have not one but 
several prototype formulas which are able 
to predict 85 to 95 percent of the variation. 
This high level of precision represents an 
improvement of from 35 to 75 percent over 
the validities of older readability formulas. 
The purpose of this paper is to describe 
the nature and results of some of the recent 

Dr. Bormuth is an AsMK-iale Professor of Education 
at the Uiuversily of ChkaKo^ 4 
• This article has heon ri prinlod frcfm the DectMU- 
Imt. issur of hlvrnvntary Eiif^lhh, Vohnne 
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research arid the efforts currently being 
made to forge our newly gained knowledge 
into practical educational tools. 

Areas of Advancement. Among the most 
important events leading to the present 
developments was the publication of two 
books summarizing the readability research 
done up to that time. One Nvas by Chall 
(1958) and the other by Klare (196^1). 
From these bo^ks it became clear that 
future ' readability research had to con- 
centrate on three problems. First, a more 
reliable method had to be developed ^or 
measuring the difficulty children have in 
understanding materials. Second, research- 
ers, had to develop a real sophistication 
in linguistics so that they cculd leum 
to meaGure and describe the linguistic 
features of materials that are really im- 
portant in affecting comprehension. Third, 
investigators had to. analyze their data in 
far more detail than they had up to that 
time. What follows is an account of what 
resulted when efforts were made to attack 
each of these problems. 

Measurement of Comprehension Difficulty 

Problem. Until recently,' investigators 
used multiple choice tests to determine the 
comprehension difficulties of materials. 
They made a test over each of the passages 
they were studying, tested the students 
after they had read each passage, and 
then found the mean percentage of ques- 
tions answered correctly. The test means 
represented the difficulties of the passages. 
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This method presented two problems, 
First, because the test, was itself a reading 
task, the investif^ator was never quite cer- 
tain whether his test mean measured the 
difficully of the passage or just the difficulty 
of the test questions. Second, these tests 
could tell him nothing about how difficult 
each word, phrase, or sentence in the pas- 
sage was. 

Construction of Cloze Tests. Shortly be- 
fore Chall and Klare published their books, 
Taylor (195:}) reported hvs first work with 
,the clo/.e procedure. The fhzc readability 
procedure can be used to make tests/roni 
any passage of written language. To do^so» 
the nivestigator selects the passage he 
wislies to study, deletes every fifth word,' 
and replaces the deleted words with under- 
lined blank spaces of a standard length.- 
The test is given to children who have not 
^ previously read the passage, and they are 
instructed* to wrjte in each blank the word 
they think was deleted. Tlieir responses 
are scored correct when they exactly match 
the words deleted, but disregard minor 
misspellings. 

Mvantages. Cloze readability procedure 
does not confuse the measurement of pas- 
sage difficulty by injecting an e.xlranedus 
reading task into the process. It also has 
the added advantage that investigators can 
measure the difficulty of every word, phrase, 
or sentence in a passage;. 

Research. The clo/e readability pro- 
cedure drew the attention of readability 
researchers who set about studying cloze 
tests to see if they were valid and, reliable 
measures of the comprehension difficulties 
of passages. Tjlieir research has become too 
extensive to/ review here, but Bormuth 
( 1967a) and Rankin (1964) have each pub- 
lished detailed analyses of this research. 
In general, the research showed that clo/.e 
readability tests are highly valid and highly 
reliable ineasures of the comprehension 
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abilities of students and of the compre- 
hension difficulties of materials. 

Description and Measurement of Language 

Early researchers felt a need to make 
their formulas so simple they could be 
used e\en by clerks having little technical 
knowledge of language. For example, to 
deterpiine the cornplexity of a word, the 
analyst either counted its syllables or 
looked it up to see if it was on a list of 
words thought to be easy. To determine the 
grammatical complexity of a sentence, the 
analyst counted the number of words, and 
sometimes the number of prepositions, in 
the sentence. While it was. at that time, 
important for formulas to be simple, the 
old formulas vastly over-simplify the rich 
array of language features that influence 
its comprehension difficulty. The over- > 
simpjification also contributed to the fact 
that the old formulas were inaccurate. 

Voatbulartj Complexity. Present investi- 
gators are probing more deeply into the 
question of what makes a word difficult 
to understand. It is a gross oversimplifica- 
tion tQ say that the words on some list have 
been shown to be easier to understand. 
This leaves as still asking which of a word s 
many meanings did children understand 
and why those, words are easier for stu- 
dents. What follows is a discussion of some 
of ' the features cf words currently, being 
investigated. 

Word Length, Children have always 
thought of long words as hard and short 
words as easy. Hesearchers have recently 
rediscovered this fact and begun investi- 
gating word length as a variable. Coleman 
( 1961 ) found that the mean length of the 
words in passages has a correlation of -.90 
with passage difficulty when length is 
measured either in terms of letters or syl- 
lables. Bormuth (1966) found correlations 
..76 and -.68, respectively, for the same 
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meaMircs. The uiffcreiuts in the sizes of 
the correlations found by Coleman and 
Bormuth probably resulted only from dif- 
ferences in the \arial)ilities of the passages 
they studied. 

Morpholoilical Covipltxitii, A word is 
often a complex structure which may be 
analv/.able into a stem and »i series of 
inlleetional derivationah and lexical af- 
fixes. This may be an important source of 
diiTiculty in understanding words. Colem*in 
{ 1966 ) found thtit passage difficulty had a 
correlation of -.SS with the mean number 
of affixes and stems into which the words 
ijji a p«issage ttin be «uialy/:ed and a cor- 
relation of the same si/e w ith the number 
of inflectional morphemes. However, it 
should be noted that this analysis yields 
almost the same results as counting the 
number of syllables in words. 

Latin Base SyUablcs. Mtiny of the words 
'ill English contain syllables which can be 
traced back to a Latin origin. Words con- 
taining; Latin base syllables gi\e'the sub- 
jective impression of being abstract 
Coleman and Aquino (1967) (7) have 
found 'Jiat the difficulties of piusages have 
a correlation of -.81 with the proportion of 
their syllables that had Latin bases. 

Ahstrdctuc^s. Although there arc almost 
as many nicaniugs of the word abstractncss , 
»is tlure are people who use it, nearly 
everyone agrees thtit» wllatever it is, it 
has an influence on tlie difficulty of a word. 
Coleman (1966) devised a definition which 
permitted hiin to count reliably the pro 
portion of nouns that referred to interiuil 
mental states and found that this number 
had a correlation of -.78 with passage 
difficulty. 

Frequency » It has long been known that 
the frequency with which a word is used 
has some influence on the difficulty people 
have in understanding it. But, frequency 
was thought to be a weak variable since 
Lorge (1919) had found only a correlation 
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of .51 between it and difficulty. More re- 
cently, Bormuth (1966) has. shown that 
fre(piency and difficulty have a curvilinear 
relationship and th<it« when this fact is 
taken into account, they have a correlation 
of .66. Klare (1967) has taken the position' 
that the frequency of a word may directly 
reflect most of the other characteristics of 
the word. 

GramvuKical Com))lcxity, The degree of 
intricacy of the grammatical relationships 
between the pavis of a sentence has always 
been considered an important soiircc of the 
difficulty people h«ive in understanding the 
sentence. Until recently, the chief means 
of assessing grammatical complexity con- 
sisted of counting the number of wjords in 
sentences. Two major objections can be 
raised to considering sentence length as 
the sole factor affecting grammatical com- 
plexity. First, it forces us to accept the 
dubious proposition that all sentences con- 
taining the same numbef of words possess 
the same degree of complexity. Thus, we 
arc asked to believe* that the sentence 
Tlic man the hoy who found the penny 
a/nV/i. ti^as lost has the same degree of 
complexity as Tlic penny v)hich the boy 
xtliom the man saw found was lost. Second, 
the number of words in a sentence does 
not measure a natural unit of language. We 
cannot simply add or chop off a few vyords 
to make the sentence more or less complex. 
Making a sentence more complex may or 
may not increase the number of words it 
contains; and increasing the number of 
words it contains may or may not increase 
the complexity of a sentence. 

The grammatical complexity of a sen- 
tence actually results from the grammatical 
structure of the sentence. Consequently, 
modern researchers are investigating* mea- 
sures of gr4mmatical complexity based on 
the grammatical str\i(?tiu:es of sentences. 
Their approach is firmly supported byjhe 
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oxiHTiUK'Hts {KTloniu d 1)\ Marliii i 19f>fi i 
and J«>lniMMi \ lUfiGa. 196(il) > wIikIi cU moii- 
straU* lluil p'M)j)lr iilili/c tin* phrase slnu 
Uirc ol sciiU iic( s as \\\v\ pron ss llic 
seiiU'iKCs. 

hi/n/m/if Dc/)//*. YngNc il9f)0) (k- 
veKuM'd an aiiaUsis wliuli oblains iniiii 
1km oI uraniuialual Luis a rrackT iniisl 
UMiipoianly liokl in his lutniorN as h( 
reads a stiilcnte. ^lAinnahh. llic nu)u 
Uranmialual Jai ls iIk- r??t*k-r uuisl hoKl in 
his iiu'iiuHA al an\ one limr as ht rt ads 
a srnU-iKf K\\v iiuirc hkt^Jy he is lo lor^t I 
ojir ol ihusc lails and ihc more hkiK he 
IS to lail lo tciniprchcntl Nuinc a.spixl ol, 
ihf si-nU iK t Mailin \ 190(> ) has shuwn 
■thai pt'cipk'N rt**|)«MiM's lo MiiUiKt's an 
rk)v-lv relah'd lo lh<- (h"])lh ineaMHts ot 
lh(* MMiUiucs. Hornuidi ( louad a 

torrt latUMi ol -.33 lu lwccn dt plh *«ad pas- 
a{jr diffirullN. KurlluK In* lunud 
dial tlu» riftiLs prmhiii'd l»v drplh wu* 
iiulfpt iidc'iil ol dn)s(' puuhutd l»\ >t n 
tt'iiu" k'liiilh. 

Motlifwr Distant t \ \anatiuii **n ih* 
()r|)lh iiicasiirr was dt'^\t hipMl hv Bi^rinttlh 
< 1967 J and is Ikmiii; iiiM stiuaU d 1)\ Inni 
aful hv. C'oh-uian and Aquino \ 1%T), This 
\.^rial)h' nuasuns the inunhn wouls 
(KViirriiii^ IVlwrtii a word or phrasi and. 
lh<- word or pliiasc it iiiodifics. on ihc 
lliror\ lhal ihc lonm i a liraminalu al Lul 
iS;ht»ltl in imMTioiy, ihi* inon- hkcK it isV 
lluil it will he lor^olltn. Prt'liuiinarv ii; 
sidls iiidi(alt- lliat du n- is a loirt lalioii ol 
-SO and hclwceii ihis fcalun- and 

passaij;f dilhiully. 

Translonudtumal Ctnniih xittj. A scnlt iik 
siuh as Tlw liitlv htnj utit may hv rv\m' 
seiiUd is rcsiilhn^ Iroiii a Iranslorinalioii 
whah einlu'ddi'd llio kernel scnUMicc Tlw 
hotj mis littlr inlo ihi' kernel simiUmicc 
77ir hoij ran, (.;u>insk\ ( 1963^ has ari^iu d 
lhaV Ui inlt'ipii'l I stnUiRc j)(t)ph' miisl 
Iranslorin ti scnlt-iK*' hatk inlo ils kt rut l 
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An inUM.sliii^ aspicl ol llio Iranslornia- 
lioii anal\>sis is llu- fad llial il can In* iisc»d 
Ut IIK asm (■ whal carU rcsi arduTs referred 
lo siil)|c( li\i l\ as luiiin llie klea deiisily ol 
iiialerials. ColiMiian (!%()) found dial llie 
proporlion (»l wtirds dial were derived by 
noininali/ini; verhs and ad)ecli\es had eor- 
ri'lalioiis of "6 *nKl -.57. re spec lively,' wilh 
passage diHiiullN. Many parK of .speech 
rcprrs^'iil Iransforinalions, also. Borinulli 
0966) loiiiid dial couiils ot llie various 
parK of speech e.vhibiud corre|alioiis as 
liii^h as .SI Willi passage difficully. For his 
current sludics. Borinulh ( 1967h) (1) has 
dt \ eloped an inveiilory of whal seeias lo 
be all die Iraiislorinalioiis ftmiul in Unglish 
and is sliuKiiii^ llu elfiils ol each Irans- 
lornivihoii on dilhriillN . 

( i^ittt ^tmd VaiiahU s. Mt>d( rn resiMrchers 
.iu lu«4m<» l)t'\ond llu wt)rd and llie sen* 
ttiut In find tin halnres ol language lliat 
rait o^» i h>nm r se^iiienls of lext lo 
inlluenK* eouipn hciisiou. Rosenberg ( 196()) 
iountl indii^alions llial pa»»ages conlaln- 
mg woids which pet)pK leiW^ lo associate 
with eath Mlu r are easier loVecall. Cole- 
man and Ai|uiiu< i 19o7) are finding thai 
anaplioiK anal; s( s \ield \ariables thai 
pn (lit I passage diifienlly. .\iiaphora ixrv 
words or phrasfs vhicli reefer back lo an 
t arln I word oi pli asf in a passage. Tin- 
ns( t)l anaphora indicales llu- exleiil lo 
whub a ])assagt' ckals hi dcplli willi a . 
singU h)pi( . Sum llu* work in lliis area 
IS onh In gmiimg. il <s siill loo (»arly to * 
prediel lis oiileoiiies, bul, il seems H^dy 
lhal gains in lliis .in a will lia\c greal value 
in iiurtasing our abilil\ lo predict and 
tonlrol passage diifienlly. 

Readabilily Fotmulas 

KaiK in\esligalors had lo delcr llie in 
vcshgalioii lA many iinporlaiil problems 
nnlil rt s( areh in cUlu r diseipHiies had made 
lools available (or .sliidyiiig lliose probhMiis. 
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As liia\ In* srrii Ironi tin prtti'diii^ dis 
tusMon^ iHiHiiislK^rVsiartli pruxkKil read 
ahililv n'MartliiTs with ruw and pouiTlul 
Umls for aiialwini; laii^uai^r. Siniilarl\. ri 
srarcli tools iHcaiiic a\adaM( for studvini^ 
the proliKiiis in\ol\cd in drsi)»iiiiij; rtad 
ahililv loniiiifas. As a ft salt. \\r lia\c' now 
Icariit d (iiotii^h to ilt si^n uuuh soundti 
r(-adal>ilit\ lorniidas. 

Ri (ultilnlity. ami RKulinj;^ Ahilittf, A prub 
Irui loiii; trt»ul>luin rcscartlnTs was tin- 
Ajvu stitin ol wlic tlii i tilt fc atun s tliat in 
lluiU(«'d nailaliility tor ^or rradtTs also 
uttliu iKt'tl till rt adahilit\ o' inatiMals tor 
nion al>l< n vtdt rs. It thi saiiit fcatnrt s ot 
lan^uai^c intlu« ua u-adal»ilit\ tt)r hotli and 
hv tin* same amount, thriF a single and 
tairlv sini|)l(* turninhi lan hi- used to prcdiit 
r(adaliilit\ fi»r all stu(lt:n1s. ri-i^aidh'ss o\ 
titt ir ItAtI of ai loniplisliiiu nt in rt adini^. 
But it diflt rt nt tt attut s infliit iKt diffiudt\ 
Un stUtK'iits ill diflt nn^ levels of riaduii^ 
at hit veinent t>r if tin same tetttures in- 
flut net* dilfuultv li\ difft rt nt amounts, tin n 
\\t musl de\(lop mt)rt ctjinplex and ma- 
tenallv differtnt kuuK t;t Ivriiuilas. Bornuitli 
ltd tins prt)l>lt ui ai^l tt)tuid that., 
regardless of tlit pt rst»n\ readmit tihilit\ . tin 
same teatures t>t lani;uai;( thttt uuist d dit 
iiiulu toi hiin (aus(d tht sanu ainotint tif 
ditlk iilt\ tor othe rs. 

Thivtatt Wiis dtMihI\ iittt rt stmt; Im (au^« 
si»ine pet>j)le lield that tjnl\ leathtr> tjt 
\tiiini; thilditn had tt) (onttin tlitnistUes 
with tht rt atlahiht\ tit instrui tioihtl tna 
t( rials. It can Ix salt] that ]ani;uai;( 

has just us strtin^ an mtltituci on what 
ailults art ahlt tt» Itarn trtuii matt rials as 
it diit s on w hat L hildn n an ahlt to learn. 

Shajtrs of tin lUhitumshiiti A second 
({Ui stion was whether or not tin* relation- 
ship hetw'vu language variables and the 
dilfuultv tit diat lani^uai^t was hiu ar, I'tjr 
i \ampli\ is tht thtlt rt ntt m tliffiuiltv In 



tween two and three s\llal)h* words as 
i^reat as the diifereiicr in difliLulty hetwecii 
se\t n and eight s\ liable words? If yot, tlu* 
simph' lorrelation teehiiKjUes used b\ tMrK 
rest arthers \uld misleading ^rc'Siilt.s, Bor- 
iiiutli* i 1966) found that man) of the rela- 
tionships slitjwed \ar\ing degrees of cur\a- 
turt . For e,\amph\ adding another .syllable 
to a one ,s\ liable word incrcMses its difficulty 
far more than adding another syll;il)le to 
a s(\en s\ liable W(»rd, The same is true 
(»t mail) other ft atures. Interestingly, it 
was the \ariabKs most treciueiitly used hi 
tlu t)ld tormulas that showed tlie greatest 
anitiunt of iui\alure. lUnee, future read- 
ability turinulas must include appropriate 
tianshirinatitjiis ot mt asurenieiits taken of 
these teatuies. 

9 \ 

FoWi of tlu Formulas, The o\6 read- 
ability forinulas wt re presentedMu trK'/t>nn 
tit what is taUed a multiple \ariabie. limjai 
((piatitJii. These etpiations lia\e a cliar- 



ttiteiistic that makes then 



unsuitable for 
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use as riadahility prt dieti6n tormulas. To 
use them we must assum * that any cor- 
relation observed between t^\o \arial)les. 
say St nteiit t K lurth and wjjrd length, must 
-jilways i\ist and Uiat it; musl be of the 
magnitude ohser\ed in the dllgiiuil re- 
st tirth. This simply is nt)t true of the Ian 
guage teatires used in most formulas. For 
t \timple. it is a simple matter Jo write long 
tjr short sentences using \vords of any 
length we choose. The result is that the 
old h)ri!mlas yii Id mish-adhig rc*sults whcn- 
i\t r the (tjrrelation is anything oUier than 
the lor'relation the formulas assume. Most 
future readability formulas will prol)ably 
b( desigiK cl to pro\ide a profile of the 
level of difficulty represeiit(»d by each of tin* 
Kiiiguage features in a passage*. ^ 

I Summary 
lUadabilitl researcluTs have made rapid 
struhs in the past fe»w years, iuereasuig 

iO 
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llie acinr*uy oi rculability loriniilas bv as 
Hindi as 75 percent. The reason lies hu^clv 
hi tlu* fact that ri-MMrc lu'rs in scAcral 
disciplines ha\e devvh)ped researt li tools 
which have aitlc»d ure.itly the stndv ol 
readal)lhl\. Ps\choloHlsts ha\e de\(lopid 
rfK' v\o/v prfac»clure into an accnratc* and 
K liahh' iiu'lhod (»l nieasiirinij lani;naye dil 
lunlly Lniilnisls have de\c-h)pe(l discrii)- 
lioiis ol \ariuns leatnres ol lanijnatte and 
these dc»seripti\e dt \k('s ha\e been lurthei 
adapted inio p«)\\erlnl new tedn'iiqnt s loi ^ 
ineasnriniii the leatnres of laiij^naije that 
UillniiiK il> toinprc hcnsioii dilficylt) 
Kinallv. ad\an(es ni «)ur understanding ol 
the n1alh<MnatKs used in onr analyses ha\e 
led to nnpro\((l designs for readahilit) 
lorinnhis. The re sult (»l these ad\aiu(*s is 
that, within a year or two, educators wdl 
ha\e platt'd in their hands powerliil new 
tools lor detiTnuiiin^ if iiistruetional nia- 
terials .ire suitable "lor S->**'' '^ith their stu 
dents. Ol gi^ih r hnig range iinp(»rtaiKe. 
we wjl gain nuuh more insight into th«^ 
( onipreliension [)roc esses an^l into the pro- 
4^('ss( s bv whiil* Knimiagc nia\ b« nud( 
int»re niiderstandable. 
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The Role of Word Frequency 
in Readability* 



C)\iT the yiMf.s ni»iiu Inindri'ds of Km 
mui^c \aritibU»s have bci-n found to be • 
prt'dk'liM'ly ri»hilt'd to nMdabilily. Thi-u-. 
ha\(' vMu iucbidcd Mitb Mciniiii;l\ un- 
likely ones iis piTcvnlagi' of words bi*- 
Hinnuig with If. /i. />, i.\nKl c. furtberinoro,^ 
those mentioned hi llu* reeenl analyses of 
Bormutli (3^ and (.oknian (10) indijiate 
that the number of such variable s is prob 
abl\ limited more In the luipn'>t's resinirce- 
fulness in (piantifyin^ new cliarat teristics 
of lanpuaHc* than b\ anything else. Ohc 
wlu) wishes ta unde rstand and eontrul read 
ability is tlurefore faced with the probkin 
of makinir sense of this profusion of" 
\ariables. 

One way to a\oid lia\ing to deal with 
suc*h an uumanaueable number is to com 
l)inV the sc\eral most hii;hl\ predictive. 
This is what the de\ elopers readabil;l\ 
formulas have clone. The conmion proce^ 
dure, of'eourse. has been* to startw ith the 
most preditll\e \ariable?then add iht next 
mo^t predieti\e. r/r.. in «i regression eijua- 
li(m. Thou^Ii ffyruuilas with eight or more 
variabUA hnw been put together in this 
way. most fonnula.s-*ha\e been limited to 
two or three \ariables. The reason is simple 
enough. be)ond this point the amount of, 
pri»dicti\e acxura(\ adJl^l be conns \cry 
small compared with tlK"^ amount of ad- 

i)r. Klin* i\ .1 Profesvur i»f PsStlmloiny uiid Dimu 
ol ihv CoWiM*' *>f \ti\ .in»i S*.R'iKC^ at ()hn» I'm 
versily, Alluiis 

• Tlnv artkK- lias Uvt u i* priritrd ir<»m lh«» faiiuarv. 
i*KiS. iwitv 111 I'M mt uUtrtf Ltifik^lt^ \ nluiiK* \LV» 
pmiN 12-22, 
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ditional work needed lor formula ap- 
plication. 

Witrd difficulty and sentence difficulty 

A look the two variables most often 
used in formulas mdicates that they are 
\ 1 i some «ispeet of word difficulty, usually 
fretjOenev. and ^2) some aspect of sen- 
tenee difficulty, usually length. Of the 31 
formulas published up. to 1960 [see Klare 
{3\ 17 use a word-eount factor directly 
and most others a related factor (c.g.j^ord 
khgtb ). SnndaFly, 12 use a sentence-length 
f«u'tor direetly and many others use it in- 
dirvetly tr,f(.; proportion^ of simple .sen- 
tences ui the sample sentences). Two fac- 
tor analyses of the Gniy and Leary matrix 
(21) of readabiliC^ variables indicate es- 
sentia IK the same two factors to be the 
most miportant. Brniton an i Danielson (5) 
found that a "\oeabulary' fc^^etor accoinited 
for the greatest \ariance and a ociilence** 
fael(»rHhe nc^\t greatest. Stulorow and New- 
man (31 } found that 'relative difficulty of 
w ords" ac counted for 34, percent and 
"relative sentence difficulty" 20 percent of 
the total \arianee of 93 percent contrib^ 
uted b\ the top ten factors they found, 
i None of the remain nig ten accounted for 
more tlhin S percent of the variance, and 
no others w-ere therefore interj i eted. ) 

No «ittempt will be made here to examine 
in detail both word and sentence* factors. 
Attention will he restricted to words, and 
in particular to word frecjuency. not only 
because it has been more ^»dietive but 
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also becanso studied of its elfecl provide a 
belter understanding of readability. In- 
cluded implicitly with frequCMiey. however, 
will be the sub-concept of -familiarity" 
as determined, for example, by Dale (12) 
and used in bis formula with Chall. As 
shown by Nol)le (14). rated familiarity is 
detennined almost unicjuely by freipiency 
( to the extent of an uid^*x of reliftionship of 
.998 in his work). ' 

Some I (dues of word frequenaj 

This first import^mce of word frequency 
is not so surprising when viewed in the 
light of language usage. It has long 'been 
known that humans do not use different 
words ecpially often, ;'ven in the long run. 
lustead. ome words are repeated very 
frequently and others its^^'d very infre- 
tjurntly. The extfr«nt of the repetition, how- 
v\It. is much greater than is usually real- 
ized. For exi^nple. French. Carter, and 
Kot^nig (15) have reported that the 100 
most irtMjUfiit words made up 75 percent 
of the total ol approximately 80»000 words 
in 50() telephone conversations. Coleman 
ill) has found that in speech the 100 most 
fretpieut words make up approximately 49 
percent of the totah It lias been found that 
though speech contains somewhat more 
repetition than wriUng. the same tendency 
diH^s inde(»d hold. Wendell Johnson (29) 
has c-ol!ected a large body of data on this 
entire phenomenon; following the philos- 
opher Charles Peirce. he has also recom- 
meiuled the use of "types" for die number 
of different words in a passage and of 
"tokens" for the total number of words. 
Que of Joluison\ studcMits, Mary Mann, 
found that 25 percent of the total of 67.200 
h)kens used by 24 university freshmen in 
writing their life stories consisted of only 
10 typ<*s ( tlii\ L and. 
and it). 

G. K. Zipf (60) has. in fact, found the 
saute geiieial trend for many different lan- 



guage situations, as has E,. L. Thorndike 
(55), among others. Zipf has referred, to 
this phenomenon as "equilibrium," and has 
attempted to (piantify it in the "standard 
curve" of English words. He has also at- 
tempted to explain it in terms of what he 
called the "Principle of Least Effort/* 
wHich he saw operating in human behavior 
other than language. The principle refers, 
briefly, to. minimizing the "probable aver- 
age rate of work" required to reach a goal 
(I.e., send" a particular message). Others 
(e.g., Mandelbrot) have suggested modi- 
fications of Zipf s work, but the basic notion 
of humans mirtimizing the energy they use 
when coinmunicating remains a central fea- 
ture, of language usage and one of the pre- 
sumed bases for frequency findings. 

A closely related aspect^ also deserves 
comment: the tendency for words to be- 
come shorter as they are used more fre- 
quently. Zipf is usually credited with first 
having shown this relationship, arriving at 
it from the observation that words become 
shorter with length of time in the language 
(time and usage being > themselves cor- 
related). Dramatic examples can be seen 
in the reduction of ' television" to "video" to 
"TV" (or "television" to "telly" in Great 
Britain); "horseless -carriage" to "car"; or 
the reductions of agency names to letter 
abbreviations such as NUA Or WPA in the 
alphabet dav.s of the Thirties. 

What Zipf showed was that this process 
occurred, in somewhat less dramatic fash- 
ion perhaps, with most words. In the pro- 
cess he gave this frequency-shortening 
principle its clearest and most detailed 
statement. Actually, like many another sci- 
entific observation, it was made earlier and 
siinpiy went unnoticed by most writers. 
Mata V. Bear (1). in an unpublished 
Master's thesis eight years earlier at the 
University of Chicago, found a close cor- 
relation between word length and fre- 
(jueucy of usage. Furthermore, this prin- 
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tipK' jcHiml c arl\ iipplitatioii in rt ad»il)ilil) 
ioriniilas. fi(Mr h^ul fouiul tlu» percrntagt* 
ol inoiiosyllalnc words in a srlcdion l«) 
pro\id(' a lair iiultx ol reading diffiddly. 
and (I li. [ohnson (25) made Oiis tin* basis 
ol Ins lornnila. 

FrrtjiH'tini and rcadahiUtij 

I'crliaps tin* t'arli(»st recorded f\itU»nc(» 
thai Ire*nu»nrv was related to rtMdal)ility. 
howe\cr. c\nne over a thousand y^Mrs 
rarlier Lorgc (36) pointed ont that the 
Talinudisls at that tinu- made word and 
idea (oinits of their manuscripts so that 
llu'v ecHihl use Iretpiencv of otcnrrenee to 
distingnisli usual from unusual nieauini^s. 
KecoguUion ol frecpic^nev's importance is 
<*orrol)oralc»d in a general way by (iniy 
(20 k who indicates^ that stiuly of the ele- 
nunl> ol dilfieiilty began centuries ago 
in ecniueclion; *'»Uh childreirs reading, 
eonerele e\ideuce pf this appeared in 
ISIO when ease* ol understanding in the 
McCinlfev Headers was considered in terms 
o\ \<)cabulary. 

hnportant in tin* early work was the list 
of 1,50{) familiar Russian words provided 
by X A, Hnbakin in I8h9. Just aftc^rward > 
in 1S9S. the German F. \V. Kaeding pub- 
lished his niannnotb woid count It pro- 
\i(hHl a seicMitifie base lor the relation of 
vocabulary to reading diflieully and estab- 
lished a basie voeabulary foundation (see 
I,<)rge), It alM) set the stage for the 1921 
appearance ol Iv I.. Thorndikes The 
Teachers Word Boot^ (57). This book 
was of .special importance because it be- 
came one of the bases not only for the 
lea(hing of vocabulary in tlu* schools but 
also l(^r the first measure ol readability that 
can reallv be considereil a fonnula, tliat 
of Lively and Pressey (34) in 1923. Two 
siunlar word books were siibsequentl) pub- 
lished by Thorndike (58),- and all three 
played an important roU* in the develop- 
iiieiilal hislorv of formulas. 



Word fri'tiucnLifasa tarmhlc 

It IS not surprising, then, to find word 
lre(pu»ney appearing as a variable in some 
way 111 most attempts at readability mea- 
surement. What is, suri)rising is that rela- 
tive *ly little* attention has been paid to a 
closer examination of how word frequency 
alfects readability. It appears possible, 
however, to gain at least a somewhat better 
understanding by examining the many re- 
cent experiments which have yielded be- 
havioral correlates of word frequency. The 
remainder of this paper will be concerned 
with such studies. No attempt has been 
macji* to provide a complete review of: 
vocabulary studies, [a goo<l reference here 
is Dale and Razik (1,3)]; word lists and 
lre(|ueiicy counts [see Fries and Traver 
( 16) and Bougers (2)]; or even al| studies 
providing behavioral correlates of fre- 
(piency [Brown (6) reviews studies re- 
lating frequency to recognition threshold 
up to about the end of 1959, and Goss ( 19) 
covers studies relating frequency to paired 
associate learning].' 

Attention, then, has been fociLSsed on 
those studies that can provide some help 
in understanding how and why word fre« 
(pieiicy is related to readability. Before 
tins (pu^stion can be meaningfully examined, 
however, a clear picture is needed of just 
what is meant by "readability." A review 
ol the many validity studies of readability 
formulas [see Klare (31)] iiidicates that 
the, readability of a passage or text can be 
operationally defined in terms of the 
following. 

l.Egiacnaj of reading. The two vari- 
ables most often used to measure ef- 



> A list of .^5 aiUlitioiial references whi^h are rele- 
vant to tin- rciuaiiuUT of this dLscuss|bn is avail- 
able^ from the author. Nolo also the appearance of 
Davis Ho\vcs\ "A Word Count of Spoteu English," 
Jouttuil of Verbal Uvniiuf* atul Vetbal Bohatior, 
5 (Deceinher, I960) 572-600. 
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ficieucv air avvrago number words 
road pVr minute (or [X^r second) and 
average number of words read per 
visual fixation (or its eonvirse. a\era<;e 
number of fixations per word). 
2, Reader judgment. This is sometimes 
measured directly by asking readers 
to rate or rank from two to a large 
number of passages in terms of reada- 
* bilitv or preference; sometimes it is 
measured indirectly by determining 
the readership of two or more alterna- 
tive versions of a passage or article. 
X Comprehension and learning. This is 
most often measured by means of com- 
prehension tests of the typical mul- 
tiple-choice type, but the -^^doze'^ pro- 
cedure has come to be used recendy. 
(This latter method usually recpiires 
' that subjects fill in correctly the blank 
spaces substituted for every fifth word 
of a passage; see Taylor (53).] In ad- 
dition to comprehension tests, mea- 
sures of amoimt learned (or immediate 
retention) and amount retained (or 
delayed retention) have also been 
used. 



Frequennj and readin^^ effinenrij 

With this ba(*kground in mind, examina- 
tion ol the behavioral correlates of tre- 
(piencv as related to readability can be un- 
dertaken. First, the reading efficiency as- 
pect of readability. A relevant series ot 
studies is that concerned with the effects 
of word freciueiicy upon tachistoscopic 
m'-MSurement of "recognition and report 
which is very similar to measurement of 
reading speed. These were precipitated by 
a controversy over the relative ease of 
recognition of neutral versus emotionally 
disturbing words. 

Historically, the experiment that seems to 
have initiated the chain of studies was that 
of Bruner and Postman (7). These authors 
found that words of an emolioiially dis~ 



turbing nature were more difficult to recog- 
nize in tachistoscopic presentation - than 
neutral words. They came to describe the 
raisetl recognilion threshold us "perceptual 
defease." A>related finding was that some 
emotionally disturbing words were less dif- 
ficult to recognize than others, which they 
described as "perceptual sensitization." The 
latter term seemt^ to disappear from the 
literature, but the former became the focuu 
for a prolonged controversy having effects 
still being felt. 

In the^ years immediately following, the 
study of changes in rt>cognition threshold 
centered about the effect <;>f personal "^hie.s 
and value-related words [see Postman. 
Bruner, and McCJinnies (48)], The argu- 
ments concerning perceptual defense 
gathered force, however, with publication 
of a study by McGinnies (40) of socially 
disai>proved. or "taboo.'' words. In his work 
he noted that such words had significantly 
higher recognition thresholds than neutral 
words, and suugested that the difference 
was due to the fact they were socially 
taboo. McGinnies. however: did not control 
for differences in word frecjuency ratings, 
nie tab(K> words ased were .suUeciuently 
shown by Howes and Solomon (24) to 
'have had much lower freciuency ratings 
tb.rn the neutral words. The threshold dif- 
ference, diey suggested, may well have 
been due to this and. perhaps, to response 
supi>res,ion. McGinnies (39) subsecpiently 
denied tlMs, on the basis of his data, and 
the contro a^rsv over the existence of per- 
eeptual detense and its causes began in 
earnest. The nature of perceptual defense 
itself is not ot concern liere. but the be- 
havioral correlates of frecjuency thus 
initiated are. This latter topic will therefore 
be the focus of attention. 



Familiariiij and recogniiiou 

The probable relationship of familiarity 
of words to recognition and reaction were 
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not coinplt'ti'l) uiikiunvii prior to lliLs time. 
BruiUT and Postman, in their earlitT study* 
inditat.'d thatjlu' familiaritv to .subjects of 
sonu' of tilt' words thtv used may liavt» 
hvvii a factor aloni; with [)crccptu»il de 
ftiuse. But about this time Solomon and 
Howes were engaged in studies which 
dcarltj demonstrated the relationshi]) of 
frequent'} of octurrenee of words in En- 
gUsh usage to recognition threshold. In one 
study [Howes and Solomon (23)]. 75 
words differing in Thoriidike Lorge fre- 
quency count were presented tathistoseop- 
jtallv. It was found that visual duration 
threshold w as an approximately linear fune 
tion of the logarithm of the relative fre- 
ijuenty with which ti word occurs in [)rint. 
The authors found product moment corre- 
lations hetvveeii the two Vtiritihlcs ranging 
from — .6*8 to — .75» the negative values 
indicating that high frequency words htive 
a low threshold »md low fretpiency vvoids 
a high threshold. Many subsequent investi- 
gators have verified diis close inverse rela- 
tionship between word-count frequency 
and threshold. 

Use of an estimate of language frci|ueutv 
such as the Thorndike Lorge tables pro- 
vide cannot, of course, ticcuratcly indicate 
the frequency vvidi which a given subject 
(rctiVler) will have seen a given word. 
Solomon and Postman (50) therefore used 
e.xperiniciitall) controlled frequencies of 
exposures' (occurrences ) ranging from 1 to 
25 with pronounceable nonsense words as 
stinuiH. Recognition thresholds oncx^ again 
varied inversely with frequency of prior 
usage. Many other invest igtitors reported 
similar results. The bettei control made 
possible by using cxperimenta^ll)' produced 
frc(juency r»ither than Ihe less precise esti- 
mate based on samples of language, such as 
ihji Thorndike Lorge tables provide, can 
be seen in a study by King-EllisI*} and 
Jenkins (.30). These authors repdfffcl the 
extreme correlation of —.99 between tachis- 



toscopic e\[)osurc time «iiid the logarithm 
of fre(piency. 

Thest studies clcMrlv suggest that word 
fre(juency. 1>) its effect on visual rea)gni- 
tion threshi/id. can increase reach'ng effici- 
ency through nu^re rapid word recognition. 
Exactlv hcxi this effect on recognition 
threshold is pioduced. however, became 
the subject of a new controversy. It was 
first clearly statecLin the work of Goldia- 
mond iUid Hawkins (18). The presumption 
jn the studies up to that point was generally 
that increased frecpiency aided visual per- 
(cption. Goldiamond and Hawkins, how- 
ever, demonstrated that it affected response 
probability or response bias. They used 
nonsense syllables with frequencies built 
up experimentally in the manner of Solo- 
mon. Instead of presenting diese syllables 
in the lachistoscope for recognition, how- 
ever, llicy presented no verbal stimuli at 
all. Their subjects were not aware of this, 
havnig been told that the stimuli were pres- 
ent but below ihresliold, and when asked 
to respond they naturally gave nonsense 
syllables. The relationship between the 
logtirilbui of freipieney of [)resentation and 
"recognition thrc»shold," however, was of 
the same order as diat found when stimuli 
had bc»en presented, as in the studies cited 
above. 

Goldiamond and Hawkins' work initiated 
ii new set of studies designed to resolve 
the visual perception versus response prob- 
ability (or bias) explanations. Conflicting 
results were found, some favoring a per- 
ceptual and some a response explanation, 
and some both. Perhaps the best conclusion 
that can be drawn is that of Zajonc and 
Nieuvvenhuyse (58); response bia^ can be 
clearly demonstrated whep no stimuli are 
present, but when they are present and 
when a stringent recognition criterion is 
employed, response bias plays a negligible 
role. Put another way, it might be said that 
fre(juency of occurrence of words can have* 
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both effects, depeiiding upon the 'heading" 
circumstanees.- Kach of these might be 
seen in the reading (efficiency aspect of 
readabiHt\\ as di>Ciissed below. 

Readahilihj as efficient ij 

Cherry {8) has suggi'st(>d that a reader 
hypothesi/os what he will see in an im 
mediately forthu)ming fixation on the basis 
of what' he has seen in the immediately 
precedmg^fixations. Thits. freciuc^noy of oc- 
currence of groups of words in such phrases 
lis "of the . . or "by means of the . . 
would provide an occasion for response 
bias to occur. That is. when a reader sees 
the first word of »uch a phrase, he is likely 
to expect the sectjud and would doubtless 
respond with it if asktul to hypothe i'/e 
what It might be. In fad, it has ha^u sliown 
in studies of do/e proc-edun and of "pr«» 
dictability" (Rubenslcin and Aborn (49)] 
that such respome bias can inde(»d occur 
In the former studies, Mibjects were 
(usually) asked to fill in Vver\' fifth word 
deleVed from a text uis indicated earlier^ 
in the latter, subjec^ts were a^skecl to predict 
each successive word of a text. Tfie high 
probability of correct response, that was 
found is 'doubtless a function of many 
factors, such as syntactic rules and familiar- 
ity with the topic, but freciuency also 
clearly plays a part. While it is uncommon 
for a 'reader actually to find words deleted 
from his text (as in thcw studies), it is 
not/so uncommon for the similar situation 
m >lhich his attention wanders or in which 



2 After this paper was written an article by O E. 
Broadbent. "Worcbfreduency Klfcct and Response 
appeared [Psychohnwal Rvvww, 74 ( Janu- 
ary, 1967) 1-15]. In it, the author argues tlia a 
response bia^; hypothesis is sufiicient to explain the 
results of the available studies. He believes Uiat 
common (fre<iuent) words are perceived more 
rfidily than uncommon because of a prior bias in 
f/ivor of the common words, which tt^mhines with 
sensmy evidence favoring the objectively correct 
word. As can bt^ seen, tins is more a re-definition 
hf ('response bias" than a disagreement w»U> the 
^' [j.effect explanation suggested here. 



a word is written or pruited unclearly. It is 
in such instances that response bias, as a . 
correlate of frecpiency. may contribute to 

readability. 

Frequency may also contribute to reada- 
bility when worck are present and are 
dearly seen by the reader. The way in 
which this effect may occur is related to 
the early finding (historically) that words 
are {X^rceived as patterns. It seems ({uite 
clear that, as Bricker and Chapanis (4) 
have stated, recognition of complex stimu- 
lus patterns such as words is not all-or- 
none. As exposure values increase, partial 
perceptions of increasing degrees of com- 
pleteness and accuracy occur, and even- 
tually a subject reports a stimulus correctly. 

The effect of frequency in this situation 
can be seen in a study by Hiiseley (22), He 
tc^sted Klares hypothesis that the amount 
qxTcentage) of a verbal stimulus 'neces- 
sary foi recognition would be a function 
of its frequency of occurrence. He used a 
"word mutilator* with a mask to succes- 
sively expose portions of English words of 
differing Thorndike-Lorge frequencies and 
.words of differing exixjrimentally-created 
frequencies. He found a relationship be- 
t^veen the logarithm of frequency and the 
I>ercx»ntage of a word necessary for recog- 
nition that was especially clear-cut in the 
case of the noas(mse words. 

Word frequency has another related ef- 
fect on reading efficiency. As indicated 
earlier, it has been shown that words tend 
to become shortened with increased time 
and usage in a language. Tliis in turn af- 
fects efficiency tlu-ougb the fact that shorter 
words have a lower recognition threshold 
than longer words (see McGinnies, Comer, 
and Lacey (41) or Newbigging and Hay 
(43)]. 

Visual fixations and readability 

This tendency appears to play a part in 
the finding of Klare, Shuford. and Nichols 
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I 32' that ihi iik astirrs ol liuinhi r iif wouls 
rt ad |)( r \isual tixation aiiil iuunl)iT of 
«L\i»nls rtad |m r sitiUitl arr both sigiiifi 
( autl\ UicK .istd 111 luoir rtadtihli as op 
pt»si d to Irss r( ttdaliK' inatrrtal. A stibsf 
qtu'iit study I unpublished i was made [o 
dcttTuiine whether ur nox the aho\e ef- 
leits Were (hie cntvchf to the latt that the 
a\iia^* woid hin;th was shortei in iht' 
inoie n adahle th.in tht K ss leadahK mi 
sn)n In this study, tuhh'd \\ hitt spat* was 
plated lutwteii the words in the nioK 
reatltihh \trsnin so that it uccu[)U'd the 
sauM tot<d hnrai sptiit as the h ss ictulahh' 
\( ision. It was luund that whih' the ntiiii^ 
Ih r oi wt»uls natl \n r fixtition vni> h>ni^er 
ihlh rt d sii^ttifiuintlv. the nuiiiht r of words 
lead pt r sriond still showed a simnfiitUil 
dtnt it iiic ia\oiini; the iin)r(* n adahh mt- 
sion Thus tin latt that liet|Ut n^^^\ ol usai;e 
trhds to piochur shoitrned words ttin he 
St rn to pla\ a part, at le.ist iiisol,il^as the 
fixation nieasnrr ol iffiLancA is LontV rned. 
But as the readini; s[>ei'd int ^tsure sIujws, 
tills IS not tht i>nlv part pla\ed h\ ire« 
(piMKA, ol two words (il rtpial h-n^tlu the 
inoif IreinKiit will still tend to hr leported 
first. 

To siuiiinau/i the i fhcts of fn ijUintN u\ 
oMiuKnti t)l Words upon tlu- readiiii; rl- 
huciii \ ttspi ( t o| ieadahiht\x the lollowini; 
iiiav lu' s.iid. The Irrcpu ncv o[ int urrenee 
of the \\i>id\ iisrd is ptohahlv a niajoi 
fat tot 111 afh^^tniij; hoth words reatl |)ei 
fixation anil pi i srcond in t( xtoal niatnial. 
As fitMjiuiKx IS liiiieasiiK nu u tises in 
leading t4fitieiR\ <irr piodtued, Tlu liiuils 
of this efh 1 1 are iii)t known. althouii;li 
studies t)l ireipe iKX and ol prat titf t ffect 
sng^i'st it is prohahlx asvinptt>tu, with the 
greatest efleet produeed h\ iiKieasint; the 
fretpienex o[ low-lieijuencv i.ither than 
highdret|uenix woids. Nor is it clear 
whether the lesult is due to lesponse bias 
or petteptual lai ihtatioiu although both 
appear likelv under teitain leadini; cir- 

\ 



uinistaim s, Freipieiiey play^ a f urther role 
through its ri'lationsliip with word length, 
siiui shorter words liaxe a lower threshold 
than hjngii unid since less of a frequent 
wt)id need be seen h)i lecognitiou to take 
plaee K 

Ficqucnaj and prcfacncc 

Siiiie word freipieiicy clearly appears to 
[)Ki\ a part iu the retiding eflkiencv aspect 
ol iisidabilitv, does it also play a part in 
the aieeptabilitv of qireference for) more 
leadable as opposed to less readable ma- 
tiTial? Relexant data here are much less 
numerous th<in in the ease of reading ef- 
fiiieucv, A number of studies have shown 
that uadeis prefer a luoie readable to a less 
leadabK" \ersioii of a passage, but few 
studu s suggest the extent to which this may 
bi due to word frecpiency. Studies by 
Swaiison (52) and laidwig (38), however, 
showed greater readeiship for text versions 
with shorter or easier words versus \ersions 
wMth longer or harder words. A study by 
Klare, Mabr\. and Gustafsoii (33) show- 
ing a close lelatioiishii) between judgments 
lif \ersions as "easy to read" and as "pleas- 
ant to read" ( tetrachoric correlations rang- 
ing fioiii .S6 It) >JT} also suggests that such 
pi ef en net s niav be based upon word f iy- 
tpuiRV nhiongli its ellect Upon reading 
efficienc y ). 

A series of sludi« s bv Ronald Johnson 
and his associates lends some support to 
this Inpothisis a well as suggests a more 
direct relationship between frecpiency and 
piLfeience. Johnson, ' Thompson, and 
Frineke (28/ found that greater frecjuency 
of t>ccurreuce t)f liujiflish words was re- 
Kited to iiicic.ised "goodness" on the se- 
niaiitic differential (to the extent of a cor- 
iclation of .63). Rated goodness, in turn, 
was 1 elated to visual duration threshold 
(and thus, presumably, to reading effi- 
cieiic v ). Tliest authors found, further, that 
manipulation of the freipiency of nonsense 
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words pioduml a systrnuitic variation in 
their rated goodness. Frincke and Johnson 
(17) also lound that the rehation between 
fretjueney' and goodness persisted even 
when pronunciahihty was held constant 
through the use ol homophones. In suhse- 
(juent studies. Johnson. Frincke. and Mar- 
tin (27) and Johnson. Weiss, and Zelhart 
(26) lound goodness related to meaning- 
fulness, and in the fir.st study once more 
found goodness related to visual duration 
thrt>shold. Newhigging ( 42 ) similarly 
found "had*' 'words on the semantic dif- 
ferential to have higher thresholds than 
"good" or "mnitraF words. 

In summary, the effects of iretpiency of 
occurrence of words upon preference lor 
a more readable over a less readable ver- 
sion appears to result from erne or the 
other oi two possible eflects. or from both. 
First, the fre(piency of occurience ol a 
word IS related to its rati>d goodness, thus 
producing a possible direct effect upon 
prefcriMice. Second, the rated goodness ol 
a word is rilatcd to its visual duration 
threshold, thus producing a possible in- 
dirivt elfect upon preferiMice by affecting 
the reading eitse of a ti\\t, since ease and 
preference are tlu insehes related. 

Fmjm ncii ami comprchvusion 

The fvidtMice thus indicates that fre- 
(piency of occurrence of words affects both 
the reading ease and pri»leienci« aspect of 
r<*adability. Does this \ariable also affect 
thi* comprehension and learning asjjcct of 
readability? .Vs lor comprehension, it was 
prt-cisily becaus^ educational experience 
indicated that mori* common words wi»n* 
more compreluMisible that the first woid 
fretjuuncy studies were made. As a corol- 
lary to. and refinement of, their counts of 
word frc(piency, I.orge and Thorndike 
(37 > extended this work to co>ints of the 
frequency of w.ord meanings (dictionary 
meanings). Once agaui. the purpose was 
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to help teachers and textbook writers avoid 
undesirable rariMes of word mei>nings (and 
thus increase comprehensibility ) in their 
writing and speaking. 

Loi^ge and Thorndike began their count 
of word meanings by excluding the 500 
most fretpieiit \<'ords. They soon discovered. 
howe\er. that the most common words have 
tin- largest variety of meanings, and there- 
fore included them: in fact, their count of 
tluse worc|s has been separately published 
and is perhaps its most useful outcome 
(35). \ 

Both Zipf 1(59) and Thorndike (54) have 
pointed out [that the number of dictionary 
mi'aning.s of a word is related to its fre- 
(piency of occurrence and to its age in the 
language (presumably an index of fre- 
cjuenev). It is logical to ask at this point 
whether or not the existence of a large 
number of meanings might not cause inter- 
leriMiciv thus rendering the more frequent 
woids less eomi)rehensil)le. As this writer 
has pointed out elsewhere, a hierarchy of 
fretpiencies exists among the various dic- 
tionary meanings of a woul in the semantic 
count. Thus, use of a highly fretiucnt word 
will most often tend to involve use of a 
highly frccinent. and presumably coinpfe- 
luMisible. meaning. 

Frrijucnnj and hwninii 

On the (piestion of whether word fre- 
cpuMicy will ch^arly affect the Icaminfi of 
writteii material in much the same w^iy it 
aflects eomi)rehensibility there 1s some- 
what contradictory information. Noble has 
made a series of studies of the relationship 
of word frecpiency, familiarity, and mean- 
ing, lie has measured meaning both as 
number of synonyms subjects can j^'ve l|0 
a word (45). which is similar to the Lorgd- 
Thorndike-Zipf definition, and as number 
of continued written assoclationj subjects'*^ 
can givi» within 60 seconds (47). In both 
cases, the relationship between frecjuency 

1.) 



'I'm Hoi i oi Woni) Km 

and meaning indices has l)ren found to he 
very high (up to an index of relationship 
of .92). Using^his indices, Nohlc has found 
that both fannharit\ of v\oids and word 
meaning (46) enhance perfurnianee in 
serial verbal horning. 

Deese ( M), on the other hand, has (pies 
tioucd whether there is inj es.sentiid re- 
lationship between nord freipienc) and . 
the measure of frei* rec»»ll. He believes that 
free recall is. iiLstead. <i linution of the 
readily a\ailable »Lssociations among high 
frequencv woids. Considering the demon- 
strated relationship between frequencv and 
number of tisso( iations. how ever, suggests 
that fre(piencv may still be the import»mt 
underlying factor. Cofer (9). in comment 
ing on Deescs position, takes the point of 
view th«jt frequency of experience pla\s a 
role hi recall through "response »»vail»d)il 
ity." This term refers to die hierarchital ai 
raiigenicnt of words in a person's rcspi)nse 
repertoire (as "response l)i»Ls," ir was cdso 
felt to be a significant factor in the word 
recognition studies described earlier). 

Summarti: frequency and comprehension 
To summarize, the effects of fre(juency 
of occurrence of words u[)On stq)erior com- 
prehension in more as opposed to less 
readable material takt s sevt mI forms. First, 
increased fre(|ut iKy itself seems to plav <i 
role, as does the tories[)onding iniuase in 
available meanings' as frequenc v imU iises, 
The existence of a liiiTttrchv tunoiig unwn 
mg fiequeneies »q)parentlj redutes the ui 
terfercMice effects diat might otherwise re- 
sult vvl^en numbers of nieanings are jk)s- 
sihle. Second, seritd verbal learning ap- 
pe<irs to he improved^ and perhaps free 
recall and other iiUMsures oi learning also. 
This latter may be due either to the pres- 
<»nee of a hierarchy of available words or to 
till' inei eased associations (greater mcan- 
' ing) of high fre(jueiicy a;^ opposed to low 
ffe(pieney words. \ 
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Ab indicated earlier, the results of studies 
relatmg word fre(jueney to comprehension 
»uid letirning have not been as clearcut as 
those relating word frequency to reading 
ease iUid preference. A possible resolution 
is suggested by some recent work by Cole- 
man (10). lie found a high positive cor- 
relation between the frecjuency of conteiit 
words in 12 pass»iges and comprehension as 
uKMsured by summed clo/e scores, as 
vvould be exju'cted. For function words 
(such .as "the" or "of), however, he found 
it high negiitlve correlation between fre- 
quencv and clo/e sc ares. As he shows, the 
ovenill elhvt of frequency may therefore 
he lost under certain circumstances. Of 
particular note Is tht,^ likelihood that this 
will occur in speech and in beginning read- 
ing. Coleman s work thus suggests that 
Viiried »iud refined counts of frequency may 
be very fruitful in the study of readability. 

Fretpiency of oc^'urrence of words, as 
this paper indie»ites, clearly plays an all- 
pervasive role in language usage. Not only 
do humans tend to use some words much 
more often than others, tiiey recognize 
more fretpient words more rapidly than less 
fre(jiient. prefrr them, and understand and 
leant diein more readily. It is not surpris- 
ing, therefore, that this v»iriable luis such a 
tt ntFiil role in the measurement of read- 
abditv. As further knowledge is gained of 
da* tirtunistanees under which word fre- 
(pieney tor iin individual ii^reases, it is 
probable i\ud understanding and control 
of readability can be still further improved. 
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Experimental Studies of Readability 
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Part I 

Stimulus Diineusioas that Affect Readability 



This paper has two mapr theses, 

1. A promising reseMreh strategy lor i^ain- 
iiii; ail lUKlerstaiulini; oF laniiuage be- 
havior IS to (olleet a hum* mi^^rix o\ 
fuuctioual relations that plot the etfeds 
upon lani;uai;e skills everted by lan- 
Ruai^e charac teristi( s. Experimental 
studies of readability will provide mauv 
of these functions. 

2. The few studies we have at present, 
crude and incomplete as they are. will 
be sufficient to make massive improve- 
ments in thi» teachini; ctipacity ol text- 
books -at least at the first-i^rade and 
preschool level. To make eipially mas- 
sive improvements at the adult level, 
however, we must study the elfect of 
huiguai^e ( baractc^ristics i\u\i exist be- 
tween sentences and between para- 
graphs. This will reipiire developinii 
efficient teebuKjues lor measuring un- 
derstanding. 

Let us say that an experimental studv ol 
readabdity investigates the eflect that a 
characteristic of prose exerts upr)n some 
reading response such as speed of rctiding 
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or degree of midersttuuling. Let us call 
the characteristic of prose a stinudus di- 
nu'nsion and call the measure of readability 
ti response meajiure. Thus» an experimental 
study of readability plots a reading re- 
sponse as a function of a stimulus dimen- 
sion of prose; it plots an S-H function. 
Linguistics and the psydiology of verbal 
learning provide a list of stimulus dimen- 
sions; word lamiliaritv*. wed lengtlu type- 
tokei ratio. »d)stnict noun ratio, clause-to- 
kernel ratio. rtt\ We also have an extensive 
list of response measures: the reader's abil- 
ity to answer (pictions about the passage, 
his ability to memorize the passage word 
lor v\ord. bis ability to fill in a clo/e test 
based Upon the passage, his ability to fol- 
low instructions prescribed in the passage, 
till* information he gains by reading the 
passage, and others. 

Thus, we alriMcly have a vast taxonomy 
ol stinuhis dimensions and an extensive 
set of response measures. W hat we need to 
provide next is a huge matrix of limctions 
-graphs tl)at plot each of the response mea- 
sures as a' function of each ol the stimuhis 
dimensions that alfects it. This paper will 
review a restricted sample of the available 
literature within an outline that will de- 
scribe how we c*m prov ide such a matrix. 

fn this paper, I will introduce several 
.variables such as spelling and phonic regu- 
larity that are usually not included in 
stiulies of readability. While these variables 
are ol slight importance in determining 



18 



riMclalulitx hit adults, tlu \ art- til lonsukp 
al)K im|H)r(aiKL in drtinniiu'ii; n adabiliU 
[or M\ \rar t)lds, A llu sis ol this papiT Ls 
that llu» most frrUK* fiHd tor rxpcrinuMital 
sUidirs «)l r('<Kl<d>dit\ ii> uuv tlutt luLUsfs 
nn firsl-i^radr material. 

Ill lacl. one purpose oi this paper is ti) 
stimulate interest in experimental studies 
c)l readability by sliowini; the inassi\e im- 
pnneinenl they could m»»ke in the teaeli- 
iiiH eap4teity ot prim»»ry reading materials. 
I liu[H» to show that t^i\en a diit»» b»»se pni- 
Mded b\ cxpenineiifal studies ol reada- 
l»iht\. sloru s tail be lonstruited that are 
simple t u<ioi;h lor a fi\e \ear t)ld to read 
alter a tew nuoutes o[ (.asual iiistruetiou— 
simple enough lo t ihible us to te»ieh pre- 
s(hoi>lers ti> i( ad at home b\ tele\isiou~> 
sunple enough to prtnith material within 
the (.oiiipeteiae ol the inentidK tUid lub 
turalK di pri\ed— Mui[)le t in)Ui;h to make it 
woithwhde to ti.uli dt at diildn n to read 
at \er\ \4Hmn a^t s and riihue the ineiit<d 
ntaidtttion tluit itsults Iroin their insuKi- 
tioii troni lan^uai^e. In slmrt, I hope to 
sh«»u that littK bouks tan be (onstrutted 
uhuh are St) t as\ to ie<id that the\ will 
ilKUii^e (lit wIm^K iittture ot the riMcliness 
prt>n^ram Un prt st hot)K rs , s« t Fi^, II tiJi 
an example ol suih ti book K 

The iharai It ristics ol an i lh t^tixe pro- 
Uraiii t>l ristauh uptui rtadtibiht\ will be 
distussed Under tliret lii aihiii;s. 

1 l ilt set ol \tiinuhw diint iisious shtiuld 
be tirnani/etl ^\s(t matit all\ so that 
wt- tciii i;ani a (.olierciit tnt;i\ie\\ with 
.1 imniinuin number ot experiments. 

2 Then should be an etphilK svste- 
iiialie arrav ot response mi»asures» iii- 
c hiding tliosi' ot mt)st interest sneli *is 
iomprehension and mronnatioii g«un. 

.3. 'Ilu- experiments should hv performed 
UjKni lelexant popuKitions—releNiUit 
poptilatiojis t)l lani;uage inateritils 
as well as rrle\<uit populations of 
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rt «ideis-aiid the experimental design 
should «dlow u,s to generalize simul- 
taneousK to all populations of interest. 

I will disiiKss the first heading in Part L 
The seeond and third will be discussed in 
Part IL 

I. A Systematic Organization 
of Stimulus Dimensions 

Linguistics iUid the ps\chology of verbal 
Karning have provided a large number of 
stimulus diinensiom tliat affect the diffi- 
cultv ol printed Kmguage. We also have a 
large arr<iy of response measnres. Olniously 
we e«miitit eollect all possible S-R functions, 
tinK a tiu\ sample ot strategically located 
(jiies. Ye»irs could be wasted by an un- 
svsteinatit attack On a problem of this 
iiiagiiitude. A systematic approach would 
first tjrg»uii/e the stimulus dimensions into 
a grid. Then the investigator c*ould select a 
lew stnnulus dimensions sjx^tted at e(pial 
steps across the grid and relate them to 
ineasuris of verbal behavior. If this pro- 
duced no coherent overview, then addi- 
tuMhd ex|H'riments would relate intcrmedi- 
ati stimulus dimensioas to the same mea- 
sures ol verbal behavior. As the grid 
becmie dotted with systematically located 
S U tunctioiis. a coherent picture would 
t merge .ind the gaps could he bridged by 
interpolations. 

To illustrate a systematic organization 
t)l stimulus dimensions, I will use a re- 
starch progr.un whose purpose is to col- 
U\{ the data that will permit. us to design 
the most readable books [X)ssible for first 
graders. 

Above and beyond the magnitude of the 
improvements that are iM)ssib(c» with be- 
ginning material, there are important meth- 
odological reasons for beginning experi- 
mental studies of readability at this level: 
( I ) In beginning reading, the skills can 
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hv aiittlvA'd iiitd [MiH tl aNMKi.itr Karniiin 
and lilt' sinipliT loniis ol coiurpl lonna- 
lu)ii. lix|K*rirm iital [)s\clu)loi;i.sls kuou luiu 
to iiuvtsurc thi'se skilK in tii»lill\ toutrolKil 
lal)orali>r\ lApiTiimnls. In later nailing. 
liout'\rr. (oiiiprclicn^ion, inionnation gaiii> 
iTilKal i'\aluatiun. i stlu tK t'ii)oyiiKiit, and 
tillitr skills iMtoiiu iniportant, \Vv know 
\m lillU* ahunt im asuring th« si- latter 
skills, mmli aii«ilKir\ work innst lu doia 
lu-lori' \\i* tan nnasure tluiu witli adeqnati* 
pri'iisuui. <2» In lH\Hiniiiiig n-aduig, tlu 
lanmug*' population is small and well dr 
fiiird. it IS P'strRti-d to a lew liuiidrc»d coin- 
iiioii words. Ill Kitt-r n-adini^. liowe\i*r, tlu- 
langiiai;i* ptipulation luaiiiu's l.ir<»« and 
poorlv ik*nn» d 

I.f»t us ori;.iiii/«' tlir stinniltis dinunsiiiiis 
that alliHt tlir rratlalulit) oi Ih liinning ina 
liTial inti» sDini* sort ol s\strinatu matrix. 
At tlie pn'sciit pri*limiiiar\ state oi (Kita 
colU*( tioii lluTt' IS littii* rtMSDii to prrf( r out 
ori^aniAition to anotlu-r. lliTt* is out*, (a) 
Till' rradcr ilistiii^nislii's tin litti-rs. (h) 
Ml' parkai^i'S iHti rs and tin ir soiUids into 
words llr also rc'ioi»iu/rs words as wlioli s. 
U*» Hr paikai^ts word. iiiU) lii^lirr gram- 
inatiial units siuli as phrasi s and tlansrs. 
(d) III" packages tlii-sc into still higlnr 
ordtT units-paragraplis. st«iri«'s, « d . 

Tlit'se lour sti'ps will si>r\i' to strp olf 
inir matrix o| sliuiulns dimiMisions in sui 
hrunt ih'tail loi prilimiiiar\ data uillri 
lioit Later, as data auuinulati\ the matrix 
will li«' graduated in finer detail 1)V m 
sorting inleriiiedute steps. 

.A Disiin*iiiishin*i the LctUrs 

It IS clear that the reailahility ol ma- 
terial lor hrsr graders tan 1>'* iinpro\ed hv 
iiiere.ismg the legibility oi the h'tters. We 
are not iree to design an eiitirel) new set 
of characters. howc\er. there are many 
type fonts in use, and surely letters from 
(iiHerent fonts can he* combined to form an 
alphabet whose characters are more dis- 
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tiiiguishable from one another than any 
existing set. 

A recent studv that is relevant to the 
lei»ib!lit\ oi bi giiming material is one by 
Popp ( IS ). l\siiig non-readiug kinder- 
garteners as subjects, Popp compart'd each 
letter with e\cr) other letter. Her result 
was a confusion matrix showing how often 
these children confused each letter with 
i\ir\ other Ji tter. Hank-ordered as to the 
niinibir of times they were cx)nfused with 
one another, the most fretpiently confused 
pairs were. d-b, d-q, d-p, b-p, h-u, 

i'i k'lj, t'U. i i\ d-lu /«-;/. /"^". «•". 

Popp presented her letters in isohUion, 
but in reading* the letters are imbedded in 
w ords and the child is able to distinguish ^ 
upward protruding irom downward pro- 
truding letters. It is not unreasonable to 
assuini *Jliat tin most ire(|uently confused 
pairs might be dilb'reiit if the letters were 
presented to the children embedded in 
pseudouords In an unpublished cxperi- 
iiient. Uimilsti in { 12) tested this assump- 
tion, but her pattern of results was almost 
the same as Popp's. Uinudstein tachisto 
stopicidK projected a stimulus letter sur- 
rounded b\ two o's iopo) and then asked 
the child to pick out that letter from a pair 
surrounded by o's {opqo). She tested only 
the pairs that Popp had found most fre- 
MueiitU loniiiscd. Hank ordered as to num- 
bi i of times the) were coniused by the chil- 
driiK llimelstein's pairs were: /)-r/, d-h, 
(/-/>. /) />. M/. h-q. i'L k'li. /i-«. d-k h-u. c-r, 
/i-i/. n-u, j'L 

Bv eombiniiig letters from different type 
fonts and by making slight changes in the 
characters, it is easy to ^nake many of the 
abo\e pairs \erv distinguishable from one 
another. Some letters may be printed in 
snitill \ersions of the capital shapes so as tii 
eliminate confusions betw een /rN. 1/-N, L-i. 
etc. The letter q is always followed by tu 
Why not print them as a ligature and 
thus eliminate conlusions between q and /). 
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If llirsf sUggl'sllOlK', \\tTt' follow ClK of 

cuursi* i\\v IclttT.s Lould hi cksigiicd to be 
luori* r.slhcdutlK pltasing^luii iii llu'ir 
al)OY«»forin. \ 

I ha\c Irsltil nuHlificalions similar to 
ihv dhow in »i mtr's of I'xperiinfiils. M\ 
most ffCiMit fxpcriimiits c-ompari'd Iradi- 
tioiial to iwprriiiR'iital \ursion.s of the follow - 
ing pairs, /m/. d-h, b-p, h-iu f-'K and 
I-/. A K'Ucr was'projtxlcd lacliislosLopically 
for .10 st'c-onds Iji'twi'cn two o\ {odo), and 
a non-riMdnig knidorgarliiu'r was aski'd to 
pick that Ictli'r from a pair siirroimded b) 
o\ {odhtn. As mmhl be expected* e\ery 
cxperijnental pair was signifieaiitl) more 
legible than its eon\ enlionnLaiJunterpart 
()>y at least the 05 level). Tin* surprising 
*^ .uing was llie amount of diffiiulty the 
children had iii dislniguisliing the eonven- 
lional p'lf and d-h. Our prestlioolers did no 
belter than chance-7«S errors to 74 correct 
choices. 

To suiiiiiiau/e the ixperimental studies 
of lei^ibilit\ as the\ alfect beginning 
reatlers. the coiiV'Mitioiial alphabet con- 
taris a number of pairs that are frecpientl) 
ctmuised by beuiiiiimg readers. It would be 
a siinpK^ matter to alter the pairs slightly so 
as to make them less eonfusnm. Further- 
more, since most of these alterations would 
sUnplj recpiire that wk^ select letters from 
several typ(» fOiits \ such as capitals, and 
lower case), almost no negalise transfer 
should occur. W'e would not he addiyg' an\ 
new characters to be learned, we would 
Simply be changing their secpu'iuc of in- 
troduction for young children. 

B. Packafliufl Sounds and Letters into 

WtmU ^ 

This step IS concerned with l(K)k-and-say 
reading and with phonics. There are cx[)eri- 
menlal studies of readability that suggest 
wavs to improse readabilit\ according to 
either ^ubskill of reading. 

Luok-md'Say Reading. W'ootKvorlli (21; 
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suminari/.ed a group of studies that sug- 
gested (hat word shapes are recognized as 
wholes. The studies suggested that words 
would become easier to distinguish from 
oiu» another if the alphabet were rede* 
signed to increase the distinctiveness of ' 
word shapes. Six»cificall\. the studies sug- 
gested that word shapes would become 
more distinctive if w^e created an alphabet 
w ith ecpial numbers of upward protruding, 
downward protruding, and non-protruding 
lellers-also with an e(|ual number of angu- 
lar and curved letters. Since 1938. there 
lia\e been a number of sporadic tests of 
Woodworlirs suggcJion (r.g., Byrne. 3). 
but no sustahied program of research on 
the problem. 

RcLvntly, in an unpublished study. Des- 
berg (10) showed that word shape can 
pla\ an important role in word recognition 
bv non-reading preschoolers. Ten trios of 
pseudowords that differed to a minimum 
degree in word shape were prepared (eg, 
occn, uosa, suam) and compared to trios 
that differed to a maximum degree (e.g.. 
(//iwr, ruyc, ccwn). A pseudoword was pro- 
jected lacliistoscopically and ten non-read- >^ 
iiig preschoolers were asked to select it 
from a trio. All ten children made fewer 
mistakes when .selecting words whose 
shapes differed to a Maximum degree (p< 
.001). The experiment was interpreted as 
evidence that a lower-case alphabet could 
be constructed that would increase the 
legibility of words for beginning readers. 
Such an alphabet can be constructed by 
substituting a few capitals for their lower 
ease counterparts and by maHing a few 
other e(jually slight changes. 

To summarize the studies of an alpha- 
bet that would maximize distinctiveness of 
whole word shapes, the studies to date 
have not shown word shape to be a large * 
source of \arlap^*c in determining reada- 
bility for adults, nevertheless, there is rea- 
son to beUevc that a sustained program of 
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research could develop an alphabet that 
would !>ignificanll\ improvr readahihl) for 
first graders. 

Length. Tlic length of words clearly af- 
f(*cls the ease of recognizing them in look- 
and-say learning. Most readability formulas 
have some measure of word length, but 
length in syllables, length in morphemes^ 
ami length in letters are all highly corre- 
lated. They can, however, be disenti-ingled. 
To the extent diat words are processed 
visually, length in letters should affecl read- 
ability. To the extent diat Avords are pro- 
cessed vocally, length in syllables should 
alFect readability. To. the extent that diey 
are processed mentally, length in mor- 
phemes should affect readability. 

In an incomplete, factorial design, I re- 
cently measured die independc^nt effects 
of length in letters, length in syllables, and 
length in moq)hemes. I tachistoscopically 
projected slv words for 1.0 second and 
asked my subjects (college students) to 
momori/e the six. Mv measure was number 
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Figure I 

Mean exposures to jnemori^alion plotted 
ds a function of word length in letters. The 
parameter is word lenulh in morphemes. 




of exposures to memorization Tlierc was no 
significant effect due to syllables. Figure I 
shows the effect due to letters and mor- 
phemes. To the extent that memory for a 
lifJt of words is part of reading, length in 
letters (visual processing) and length in 
morphemes J m ental processing) appear 
more im|wrtam\than length in syllables 
( vocal processing 1 

Frcqucnci/, Tlie frequency with which a 
word is Used affects the ease* of recognizing 
it. Klare (14) has provided a convenient 
summary of the eflccts of frecplency on 
readability. 

It need only be added that the usual 
word counts may be deceptive for first 
graders. For first graders, the frequency 
counts should be based on speech, not 
printed material. Tliere are certain words 
that occur frequently in print ( it///, would, 
did, am, etc.), but seldom as whole words 
in speech. These words usually occur hi 
reduced form (7/, V/, etc.), and tl^ey 

may be difficult for the child to recogni/.e. 

PJwiiics: Lcdming the Sound-Letter A6- 
sociations. All readers sound out words or 
parts of words, but this reading skill has 
more influence on readability in beginning 
reading than in that of later years. Before 
a child can sound out a word, he must have 
learned the sound-letter associations. This 
is ordinary paired associate learning, and 
according to the usual three-stage analysis 
( Under\vood and Shult/, 22), the degi;ee to 
which a child learns any particular sound- 
to-letter association would depend upon 
three factors: the legibility of the letter, 
the response strength of the sound, and any 
previous association between the two. 

We have already discussed Hie legibility 
of letters in A above. There are several in- 
dices available that measure the degree to 
which the child has mastered the phonemes 
of English. One measure would be studies 
such as those of Tcinplin (21 ) that measure 
the age at which the child first uses the 
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plioiieiiuN (iailmlli* aiitl Jom-s \ \> lia\t 
pruMckd a stxoiid im aMirc. T1k'\ lallu il 
iir<|mMKV with whicli IIk- (hlkriul 
phoiu'iiu's ot cuiu'd III chikln n's spm h. 
Tlu'V iuiuid that thi- cuiisoiiaiit phoni im n 
oitMiriril in the lolhm iiit^ Drdcr whin 
raiiUnl ironi hiiihi'st to lowest, /i, /, ,s, 
(/. u . i L viv. 

Ill an or(hiiar\ patud associate kaiiiing 
task. lk'idiaiisk\ i I ' iiu-asurt d thi* imsi' 
with whuh ii()U-riadiu<; pu'sihoolrrs 
kMriK'd <i iiiindur of suund-to-kttii as- 
sociations, thttt IS, hi' iiKMsnri-d the tAsi 
with which thc\ kariiC'd to pruiu)nncc' icr- 
taiii k'llt is. As iniiiht hi- c \pi cti d. hi* k)unil 
a stionu lu-i(ui in V cfkct, the chikl karned 
ihi- proniUK iatioii to he associated with ii 
certain k»ttei \er\ easik il the prununcia- 
luMi was a phoneme what oecuned ire- 
(|iientK m ICn^Hsh. 

IkTchansky's experiment sug<4ests that k)r 
begmnmg readers at kast, a stoiN might he 
more reaihihk* if the pronunciation of its 
k'tlcrs w ere ircijiieiitk occuning phonemes, 
Bekire discounting phoneme control as iin-v 
practical, one should reineinlur tliat the 
\ocahular\ oi liegmiung material is tightk 
conlrulled. Just as we control voCcd)ular\, 
we can also control phonemes. A gifted 
cartoonist can write a stor\ with onl\ three 
or torn words. II the letters and phonemes 
as well as the words were carefully chosen, 
the story could he read hy an average four 
year old ( see Fig. II ). 

This Is not to sa\ that Herdiaiiskv's ex- 
periment provides us with all the data we 
need for phoneme selection Among other 
things, we need to know if a child will 
induce the basic phonic concept (sounding 
out new words) more casilv if his ht - 
ginning reading nuiterial is restricted to 
certain classes of phonemes. I' or instance, 
sounding out an unfamiliar word would 
seem more difficult if the beginning ma- 
terial consisted mostly of stops and afTri- 
(Mlcs {}}, (L /, and ch) because 



thc\se sounds are most dilFicult to pronounce 
m isolation, The child inevitably loams 
sounds such as pull huh, etc. If Bcrdian- 
sk\'s expiriments were supplemented by 
a do/en or so similar ones, wh» would be 
able to practice phoneme selection in be- 
ginning reading materials as we now prac- 
tice vocabulary restriction. 

Phouu s. R(}iul(iritij, One of the subskills 
tj| rciiding is the ability to sound out words. 
For first graders in particular, it is rea- 
sonable to aysumc that readability of ma- 
terial will \ary according to the degree of 
phonic regularity in that material and also 
according to the particular, phonemes per- 
mitted in that material.* Currently there are 
two major svstcms for regularizing English, 
the Initial Teaching Alphabet (ITA) and 
Bloomfiekl\ linguistic system. iJoth have 
considerable flaws. 

Tke Initial Teaching Alphabet must mis- 
spell To** of the words in English, and 
about 40? of these changes are radical 
ones. There are se\cral reasons for this, 
but the most profoun J one is that the .pro- 
nunciation of many of the Englisli mor- 
phemes changes if the morphemes change 
en\ironment.' Consider Hie different pro- 
nunciations of the past tense morpheme in 
trottci/, pass((/, and crosstv/. Consider the 
k)llow ing dernations. democrat ic-democ- 



>\t()ri» u''»cTally, when one fits an alphabet to a 
laiijjuajje, there are three factor^ that must be con- 
sulrrcil. Fir>t ks letter-to-souiul regularity, phonic 
regularity. But there are two otliers of almost equal 
miporlance. Second is inprpliome regularity; mor^ 
phenies should be spelled the .same way in dif- 
h reiil environments ( trottc'J, passcJ, crossed, sign, 
M'^iMifieaiit. etc). Third-and this becomes a criti- 
cal factor m a Woddwide language such as English 
-people tliat speak widely differing dialects should 
hi able to read the same language; a Kentucky 
lullhilly and a Cockney dockworker should be able 
to read the same books. One must compromise be- 
tween these three factors in fitting an alphabet 
to a language Clearly, the problem is much more 
difficult than designing a character for each 
phoneme of English; it requires wisdom plus a 
considerable linguistic competence, 

2P. 
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ran/, cximliti -i xpulnut, \iiltiifu ant, 

III short. th( iiuss|H Ihiius of tlic Initial 
Tt\uliiiii; Alplukh* t i^uarantir a <.t>iisicltT- 
ahlc amount of iKi;a(i\( triUisicr ulicn tlu- 
cltild IS sinftcd to trtiditioutil oitliOgrii|)li\. 
An t'xtt'iisut St ru s ol lii;litK controllcil 
Ial)or^itor\ t rmu nts on transit r liavi 
sliowii ioncIusj\il\ that n*i;ati\t' tian^ftr 
ottnrs anu timr »4n oriianj,sni Itains a n 
s]niiisi to a stimulus thtit he must later un- 
Irarn in i)rJ( r to suhstitutt ttnothrr rcsponst 
to that stimulus, Ol couisr, the Initial 
Tta<hini; \l|)hal)(t Ikis proMd to l»r an 
I'M t Ih nl U\{i hnit; U i hiu(|iu- so it also giuir- 
aiit<.(s a ct^iisuh Ttihlt tUiiiiunt ol |)i>sitl\(' 
transh i. tUnl tlu \\i ii;ht t)f r\i<.hiKi' sui^ 
^rsts that thi positivr trtiiishi niort tluin 
outwcisihs till lU i^titiM- trtUisft r \»'\titlu 

V 

k'ss. wc shouhl n*(lim» thr lU't^titur ti tTisIrr 
as muth as possilih . wf i ould rttlXai it 
toiisidt ral)l\ il ut uiarr\ tlu Initial Tiacli 
iiii^ AI]>lialKt to Bloomfit-lirs lini>ruistu 
system. 

In addition to its tht iirt tiuil (ontrihu- 
tioiis. Bloomfit'Urs lini^uistiL s\strm <2; luis 
shown that it is |M)ssi1>K to si ltxt a larm' 
iiuhiIkt ol F.nt;lisli words that tire spt lied 
rrgidarly. that is% that are spelled witli in- 
ternal eonsisti n( \. But his liii^nistiL s\ stein 
siilfers Irom on* fatal IKiw. iiian\ of the 
iiujsl iin[)ortant wortK in nn^hsli are not 
spelled regularlv [ Htn\ uauhl, shouhl, 
etu K Thus, if one lestriets himself to 
Bloonifiehl's list of words, he restricts his 
reading matdials to siuh senti ntes as "Faf 
Pat it on tlic ihut" Tlu rnosf phlegmatic 
child can (]uickl\ he reduced to Insterics 
if he is foreed to read too niiu h of this. 

Thus, we have two systems for n-gu- 
lari/iiig r.nglish, each with great aM\an 
tages. each with great disticKtintages.^ J'\)i 
tunateK, it is possihle to comhiiie them in 
projKiitions that ina.xiim/t the ad\aiitagt s 
of each and inmimi/t* t)ie disadvantagts. 
One sinipK spdK a U\\ do/en ahsolutilv 



iiecrssai) fuiu tioii words in an Initial 
Teachipg Alphabet and selects the remain- 
d(r of liis words from BloomfieUrs list. 
Thus, negati\e transfer is reduced to an 
tihsolute minimum. onK a few dozen words 
art miss[)clled in the transitional alphahet. 
Furthermore, it is possihle to select those 
f(w do/en words so that their misspelled 
shape in the transitional alphahcl will he 
almost identical to theii sha]:)e in the tra- 
ilitioihd orthograplu. Thus, bv combining 
the Initial Teacliing Alphabet and Bloom- 
field's linguistu list ill the proper propor- 
th>ns, we can get afinost all of the advan- 
ttiges of each and almost none of their dis- 
adCaiituges. Furthermore, if ue restrict our- 
si Kes to the most common words in Bloom- 
fit Id's list, wi- can get almost all of the ad- 
\tUittiges of the b»isal readers and almost 
none ol their disadvantages. 

Vhonks. PJumcmc Sclcctioiu We noted 
that insofar as leading involves phonic skills 
-st)uiiding out words— rcadabilit}' would 
be affected by phonic regularity and by 
the jiaiticuKir phonemes^ used in the ma- 
te riah. We have discussed phonic regu- 
liirit\. Let lis discuss improving readability 
h\ rtstricting the phonemes used in the 
materials. 

Fxei pt for Berdiansk\'s experiment men- 
tioiH d t arluT. there is almost no evidence 
that tells us which letter-sound associations 
tire easier to Ictirn tU curding to any reading 
suhskill. II we tire to mtike beginning read- 
ing nititeritils more readable by phoneme 
seleetioiK w( must guess from a very in* 
adeipiate data base. Except for the fre- 
i|ueiic\ counts of Carterette and Jones, 
tiboiit the onl) information we have are 
two r^^ugh observations. ( 1 ) It is .ilmost 
impossible to make an isokittnl souncl that 
resembles the stops and affricates {p, /), A, 
(/. A, g, /, t^ul (/i). (2) On the other hand, 
lUtinv of the eoiitinuent,s itui be closely ap- 
proxinitited in isohition, and in addition, 
thev plav a meaningful role in the Ian- 
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^ guage {S'S'S is the sound of a snake or of when we see something good to eat, ee- 
gas escaping, m-m-ni is the sound we make ec-ee is the sound we make when we are 





Figura II 

Exccrrl'* i^om a preschool reader that illustrates a technique that can used for teaching phonics 
to five year olds when the component sounds of words play a meaningful role In the story. 
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frightrnrd t7(.), Basicl uii th(s( two oh 
scr\ations (to thr cxttiit tlhit rcadiiig iii- 
\ol\vs iliv phoiik subskill l)t i;inniiin read- 
ing niatiTials might In- more nadal)K J» 
tlu'ir phoiuiut's wcrr rrstrittrd to thi con- 
tiniirnts. TIrii thr first words in tlu hv- 
ginning niatcrial would hi- oi»i\s sutli that 
their component sounds couKl phiv ti ine»in 
ingful roK' in the stor\ otC uiu, at, ett.). 
Tliis aUows a nuniher of technic jues for 
teaching phonics and spelhng that are not 
availai>le if the heginning materials inehide 
NUeh words as come and look (see Fig. ID. 

Phonics: BIendm<i. Most first-grade 
teachers wouhl agree that phonie hlending 
is one of the most diflieult skills to teath. 
Bhnding is hss (hfheult if the first words 
are composed of sounds in which the con- 
sonant and NOW lis are [)roiu)unced with 
the tongue and other «irticul<it()rs in roughly 
the same position^ the hlending transition 
from one to the other is relatively slight, 
there is little difference hetweeii the two 
s(juiuls pronounced in isolation <ind the two 
blended into a word. Sounds such »ls /s, 
sh, n« ee. a, i/ fonn words such as ati »ind 
U'C tliat'iire easiest to hiend. Compare the 
blenchng transition fiom.Nowel to conso- 
nant in ^hc \ersus tlutt in out hy pro- 
nouncing them Jowly and coiiipaiing the 
relative motion of tongue and hps. 

Let us summarize the important ponits of 
this section, reading is a complex, inter- 
locking hicrtirchy of skills, »uid lUttte rials 
that are most rcadtible ticcordiug to one 
suhskill are \er^ unreadable according to 
other subskills. It is possible, however, by 
simultaneously considering all the subskills 
when one selects his materials to get a set 
of materials tluit approaches maximum 
readability according to all subskills. Wc 
can get the advantages of phonic regu- 
larity by using Blooinfield's list of regu- 
larly sjxiled words. Wc can get the ad- 
vantages of meaningful, interesting seii- 
tcpc^es by supplementing tlhit with a very 



small number of function words misspelled 
in a transitional alphabet {tlicc, iz, wuZy 
etc.). Furthermore, wc can gain most of 
tlu advantages of the basal Veaders— the 
( fleets of freijuency— by restricting our se- 
lection of rcguKirly spelled words to those 
tlhit oceui most freijuently in English. Per- 
haps -a I though this must be verified by 
iidditional experiments— one can gain still 
further advantages if he restricts the pho- 
nemes iii his beginning reading materials 
to certain coiitiiiuents and to words whose 
component sounds can play a meaningful 
role in his story. 

\n ex<implc m^y be luipful. In selecting 
the most readable set of materials for a be- 
ginning reiidiiig program being developed 
at Southwest Regional Laboratory (see the 
e.\»unple), we used the following proce- 
dure. We punched into a computer all the 
reguliirly spelled one-, two-, and three- 
phoneme words of English. We also 
punched in e»icii word's fretjuency of oc- 
currence according to its Rinsland count 
for first graders (19). We assumed that 
our first niateruds would be most readable 
if the phonemes were restricted to the 
following phonemes. A, m, cc, /, fl, i, n. 
//*, .v/i, and //. Then we asked the computer. 
"'Which phoneme should we introduce 
next? Which phoneme will combine with 
the present phonemes to form a maximum 
number of common words?" The letter w 
Ciin be combined with these letters to add a 
nuiximum number of common words, and 
this is the letter that should be used in the 
next hook. Adding u to our letters, we re- 
pctitcd the (piestion to the computer. Con- 
tinuing these steps gave a seijuence that 
enables our books to illustrate the concepts 
underlying spelling and phonics with a 
minimum of instruction in letter-sound iis- 
sociations, each letter -sound association in- 
troduced in a book gives the child the max- 
imum payoff in useful new words. 

The language characteristics considered 
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thimble is shiny). The direction of the ru- 
suits was not surprising, the scMilenccs 
whose content words had high strength 



thus hn- are most nnportant ni deternnning 
readability for hegniners, the ones that loU 
low are important for adults also. 
C. Packa^in^l Words into Phrases, Clauses, 

and Sentences, 
* The greatest number of experimental 
studies of readability lie under this sub- 
heading. Let us subdivide them according 
to Ilockett { 13) into those concerned with 
content morphemes and those concerned 
with the framework of function morphemes. 
That IS. in the following sentences the con- 
tent morphemes are in lower case letters 
and the framework of function morphemes 
is in capitals: 

TIIK man operatKD THE boat skill- 
fulLY. 

Tllli man's op.TaTION OF THIi boat 
WAS skillful. ' 

Other sets of conti»nt words that Ciin fit in 
the same framework of function fnorphemes 
are boy-explam-problem-cpiick. tCiUn-ex- 
caViite-tuniiel-speedy. 

Content Words. In a sustained series of 
studies. Decse has shown that words 
an* organized in the mind into highly sys- 
tematic structures. That is. words and sets 

oi words are associated with one another Many studies can fit under this subheading, 
and each word is more* tightly associated and most of them can be described as coin- 
with some words than witii others. Words parisons of grammatical transformations 
an* orgam/ed into word classes, grammat- 
ical classes, and semantic classes. 

Deese's students have shown that lists 
of words that are corrshited with this men- 
tal organiAition »tre much easier to learn 
and inemon/e than lists that are not (r,g.. 
Miller. 17, Weingartner. 23 k Similarly, if 
the content words of a sentence correlate 
with this mental orgam/ation, the sentcnte 
is easier to learn or read (C!oleman, 7). In 
this experiment, noun-if.-ad)ecti\e sentences 
{ The thimble is bright ) whose noun and 
adjective had strong pre-experimenttil as- 



tissociations were much easier to learn. The 
magnitude of the results, however, was 
somewhat surprising. After a single expo- 
sure, the subjects would remember 14.3 
responses out of 24 jxiired associates if 
there were strong pre-experimental associa- 
tions between noun and adjective. If there 
were no pre-experimental associations be- 
tween the noun and adjective, they could 
remember only 1.58 responses; there was a 
ratio of 9 to 1 in relative learnability. 

To summarize, we can make tremendous 
gains in readability if we design our ma- 
terials to correlate with the way the words 
are organized mentally. Tliis could be ac- 
complished by using cloze tests. That is, if 
the majority of subjects in a cloze test disa- 
gree with the original wording, it would 
suggest that the wording should be changed. 
Giving cloze tests is an expensive operation, 
but the expense might be justified for ma- 
terials that are used very frctpiently such as 
elementary reading material. 
Framenork of Function Morphemes. 



These experiments are concerned with a 
number of the factors that are usually in- 
cluded wlicn prescribing rules for writing 
readable material. In fac^, most of the time 
when a writtr applies rules for improving 
readability,* he is actually choosing one 
gnunmatical transformation above another. 
Thus, the rule, fo r wri ting readable ma- 
terial could be stated^hwre precisely in 
terms of grammatical transtbrmations. 

Tlu* reader of* this papi*'r can verify this 
for himself by noting the number of times 
he makes grammatical transformations as 
socMations were cJinpared ' to seiitc'nces he revises the following sentence to make 
whose noun and adjective had weak{T pre- it more comprehensible {An inclusion of 
experimental associative strength {The this is an admission of its importance). If 
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he tricxS to make this suiiltiitc more read- 
able by increasing the number of personal 
words, decreasing clause length, or by ap- 
plying almost any of the rules for improv- 
ing readability, he will discover that the 
change he frequently iriakes is a grammat- 
ical transform.ition-a grammatical trans- 
formation that operates tipon entire clauses 
and .liters niaiiy of these variables simul- 
taneously, Tlie point is this: when we con- 



sider the actual operations we perform to 
improve readability, it seems that gram- 
itiatical transformations are fundamental 
units and that describing some rules of 
readabilit) in terms of smaller units is more 
or less artificial. 

To save space, a sample of the experi- 
mental studies that seem most closely re- 
lated to readability will be summarized in 
a table. 



Table I 

Sample of Experiments that Studied Graininatical Relations 



Reference Grammatical 
Number Rchtion 

Noininalizatjon 

5 vs 
Active Verb ' 

Nonilnali/ation 

6 vs 
Active Verb 

Active 
8 vs 

Passive 



II 

15 

8 

20 



Active 

vs 
Passive 

Depth 

Einbcdilcdncvs 

Order in a 
N IS N 
Sentence 



The prescriptions for, readable writing 
that are given by the ttbove experiments 
can l)c suininari/,ed briefly, prefer gram- 
matical transformations that give short 
clauses and use active verbs, prefer gram- 
matical transformations that do not use 
abstract nouns iioininalized from verbs. 

In the experiments of Table I, note that 
grammatical transfomiations that contain 
abstract nouns are less readable than their 
couiiterpiu-ts containing active verbs. The 
lesson to be gained from this is that ab- 
6tractness is not necessarily an inalterable, 



Rcspome FUuhnt* 
Clo/e Score Active verb was most readable. 

Retail Active verb was most readable. 

Recall Active was mosi: readable. 

Verification Active was most rc»idable. 

Recall Sentences of low deptli were most readable. 
Retail Readabilit) increased as embeddedness decreased. 

Recall Term at end is better remembered. 

iiiherent stimulus dimension. As often as 
not, complexity due to abstractness is super- 
fluous complexity, as often as not, the num- 
ber of abstract nouns in a pass.ige can be 
reduced without changing the content of 
the passage. When a passage contains a 
large number of abstract nouns nominali/.ed 
from verbs (operation), the writer can 
easily transform them to active verbs 
(operate). For example. The exclusion of 
that candidate is emphasis on the impor- 
tance of our group includes three abstract 
nouns. Let us transform it to a version that 
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includes none. When tlutj cxdtuh d that 
cmuluhtt(\ It emphasized that o/zr «ir(»///^ uas 
important Clearly llu' .ilislraetuess ol the 
firsl version was not due-lo niluT(*nl ab- 
^ slraclness of the* subject matter. 

In short, tlicre is piosi* tliat is abstiaet loi 
no better reason than tliat the writiM cbosc 
one (lerhalne ol Ihe \rrh instead of aii- 
otlier. 

Xote tbiit tlie abo\r disenssion ol ab- 
stract nouns nonunali/ed lioin verbs is re- 
lated to the ad\Kv thai leadable writiiijn 
sluHiid contain a lari^e nund)er ol V(»rhs, 
Note also that this does not necessariK 
mean that the writer should invigorate his 
proM" VMth verbs hkr exeojiatr and fulmi 
nate The propoilion ol vi rbs can be in- 
creased bv translcJiimnt^ abstract nouns 
noiranali/ed Irom verbs to active verbs 
PcThaps another example* will he- worth 
while. Consider the sentcnee liom the sec- 
ond para«4raph ol this se ction: Am mehi\i(m 
of this is an admission of its imjnntanee 
\Vc* can rediKc- the propoilton ol abstiatt 
nouns and simultanc ouslv iiic rc asi tin pro 
portion ol verbs hv tran.sfc>rnnnLt it as lol 
lows. S/«f( she imluded tliis^ sh( is ad 
mittiwj^ that it uas iniprntant. \oti (hat tiK 
transloiiiialions altc i luanv olhi r stimuhis 
chilli nsions usnallv uahicKd ni aclviu lui 
i< aclable vv i itinn-shoit clause s. pe isoiial 
pioiiouns. and word In (pie uc v . 

It Is ini|>oilaiit to note th.et in this ease 
the acl^lltioiial e \pbe line ss ol the ve ib dot ^ 
not come honi its slem-litMU its eontenl 
intuphi UK - but liom its luiKtion tnoi 
phe*mrs, Ti iisi . voue\ luood aspe ct, iium- 
l>ei. and peiM)iiare ivplicitlv state il bv the 
milen tional .dhv s. pioui)UUs. tUiel auxditirv 
verbs Whi'ii ue transbunie el ifubisitm to 
mthid(d. tense, voiie. aspect, and inood 
V\en- made spe c ifu . More |Lje iie itdlv . the 
finite verb is lar iiKiie c^xplic it than the ab 
strac t noun 

( ousicKt tin* moi»» readable* i»rainm.itKal 
tiaiisloiin.ittoiis ol oui cAainples above. 

o 
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Ne)le that thev contain more personal pro- 
nouns than tin' less rcMclable translonna- 
lions Much ol the abslnictness in scientific 
writiiiii must be attributed to a reason no 
more prolounci than its tradition against 
* r* tUid "wc" and its Iraditicm ai^ainst ad- 
drc^ssiiig the reader as "von." The* scientist 
who eschews the first two pronouns is 
simply beiujii modest, but not nsiiiji^ them 
lieepientiv causes him to snhstitule an ab- 
stract noun ( exphmation ) lor its activ e verb 
loim ( exphiin ). to substitute a passive verl) 
loi an active- verb, to siibslilutc* an em- 
be elded se iitc'iice lor a uon-embedded seU- 
tene c* 

More* i^eneiallv. il vvc« had more expc^ri- 
mental studies ol readability such as the 
ones sumiuari/ed in Table I. many ol the 
rules loi writing readable mate«rials could 
be pu cisedv stated in tcnns ol ji^raminatital 
tiauslorniatioiis. 



/)■ Patkaflinf^ Chinsi s into Ilifihei Vnits 
Sill h as Paraffraj)lis and Stoiies 
The re is not nmch of tUi experime ntal 
Utttme to sav heic. Surciv most of us be- 
heve th<tt the ma(oi deteimiiic rs of rctub 
.dniitv foi <tdults^ he at this level— lie in 
the (issodatioiis between clauses and paia- 
i^uiphs. m the oveuell or<4tmi/.etioii-bnt psv- 
clioloi^ists have not vet lelined the experi- 
mental technupies to invc sti^atc this level 
*encl Imi^uists are not vet .eblc" to desc ribe it, 
A.S far as psycnology'.s share of the prob- 
lem is concerned, much of it reduces to the 
problem of developing an efficient, eco- 
nomical measure of comprehension (see 
subheading D in the next issue). Given an 
economical measure of comprehension, we 
can mass produce experiments at this level 
even though we lack a refinc^d linguistic de- 
scription. In fact, mass producing enough 
exp(»riments at this level could provide 
linguists with the data dicy need to develop 
a suprasentence grammar. In the second 
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half of this paper will discuss developing 
a set of response measures. 
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Part II 

. Measures of Readability and Relevant Populations 



2, A Systematic Array 
of Response Measures 

We have seen that reading is a complex 
hierarchy of subskills, and that materials 
that are readable according to one subskill 
may be unreadable according to another. 
Thu.s, one of the most imjwrtant steps in 
making a .systematic study of reading and 



readability would be developing a sensi- 
tive measure for each of the subskills and 
plotting the R-R functions that relate these 
measures to one another. 

The important point is that different 
reading subskills are optimally facilitated 
by different stimulus dimensions. First 
graders, for instance, will find that some 
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words are easy to read by whole word 
memorization and that an entirely different 
set is easy to read by sounding out. This 
explains the ease with which an exfxjri- 
mentcr can "prove" that an extreme teach- 
mg system that emphasizes one skill is 
superior to all vothers. All he has to do is 
select a language sample that optimally 
facilitates his siibskill. 

If his system depends mostly on whole 
word learning, for instance, all he has to do is 
select the most common words in English. 
Beaiuse their response integration is high, 
a book made up of these words will be 
very easy to read as long as the child is 
depending iiix>n whole word recognition. 
On the other hand, to the extent that the 
child has to depend upon phonics, the book 
will be very unreadable because the words 
are almost all spelled irregularly. 

If another experimenter wishes to show 
that a phonics system is superior, all he 
has to do is select regularly spelled words 
that are generated by a minimum of letters 
(iW, mill, sill am, Sam, mam, mass, ass, 
sass, Sal, Al, lam, etc.) The child v.'ould 
have to learn only six or seven sound-to- 
letter associations so these words would 
be easy to sound out. However, they would 
be very difficult to learn by whole word 
recognition because the twenty words 
would be so similar that it would be diffi- 
cult to discriminate one from another. 

Simijar observations can be made for 
adult reading. Styles that are most readable 
for light reading may he very inefficient 
for material that nu»st be more completely 
mastered. 

Clearly we do not want to construct 
hooks that optunally facilitate one suhskill 
to the detrunent of others. We want to 
construct hooks that optimally facilitate a 
combination of all subskills. At the ele- 
mentary level, we already posst>s^ the 
teehni(|ues for accomplishing this, ancl they 



have been described elsewhere (Coleman, 
3). That paper described the development 
of a twcpty-word pre-primer that was 
extremely readable as measured by any of 
the subskills, ^ 

But even before a child finishes reading 
his first pre-primer, we become concerned 
with true understanding and must consider 
the relation of the prose to the universe 
it describes. Almost everyone-teachers, 
students, psycholinguists, and even experi- 
mental psychologists-are really interested 
in understanding. We must develop a mea- 
sure for it, and relate that measure to other 
measures, such as word-for-word memori- 
zation. In carrying out an experimental 
attack on readability, one of the most 
important steps is to develop a systematic 
set of measures at this cognitive level. Let 
us consider five such measures: (a) an- 
swering questions, (b) cloze tests, (c) 
verification, (d) performance, (e) informa- 
tion gain, 

A, Answering Questions. 

This is the measiu-e that has been used 
most frequently to measure a reader's un- 
derstanding of the, passage. The difficulty 
with this measure is that an investigator 
can ask easy questions about difficult pas- 
sages and vice ver§a. Since the questions 
that are actually asked are only a smi^ll 
sample from among the ones that could 
be asked, the questions that one investi- 
gator would use might bear little resem- 
blance to those that another investigator 
woukl use. 

B. Cloze Scores, 

The disadvantage of cloze scores is that 
they do not appear to measure the long 
range associations in the passage. They 
are essentially measuring the short range 
constraints within phrases and clauses and 
not the more important ones between stm- 
tences and paragraphs, 
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C. Verification. 

In st'\or«il studies, Gough (4) has opera- 
tionally defined understanding as \erifica- 
tion. lie showed hi.s reader a picture of 
an event and a sentence. The readers task 
was to say whether or not the sentence 
Iruly.described the event. 

D. Perfonnana». 

It is easy to measure the understandnig 
of a passage if the passage assigns the 
reader a task to perform. The test of un- 
derstanding is simply to have him perform 
the task. Coleman (2) used such a measure 
of understanding in a set of five experi- 
ments. I lis suhjeets were assigned mathe- 
matical tasks. Mathematical tasks are 
con\t iiieiit unrs sinn it is (.\ls\ to generate 
an infinite number of tests and tasks. 

In measuring understanding, wc become 
concerned with the relation between lan- 
guage and events in a real world. The 
relation will clearly \ary with the com- 
plexit) of the language and also with the 
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c^oinplcxity of the universe of events that is 
being described. Since the subjects were 
assigned mathematical tasks, their universe 
was a matrix of numbers such as the one 
in Table I. As their language, they were 
assigned a miniature language such as the 
one in Table I. 

Thus, wejhave a miniature language and 
a miniature universe. We can describe them 
precisely and we can vary them indepen- 
dently. Despite the restricted vocabulary, 
the miniature language in Table I has most 
of the essential features of complete En- 
glish. It permits one to study most of the 
variables that affect readability and it per- 
mits the experimenters to assign the subject 
an infinite number of tasks sudi as "Sum 
tlie even numbers and square the sum'* 

The following three experiments used 
essentially the same method and the same 
measure of understanding. Sentences that 
assigned the subject mathenutical tasks to 
perform were prepared for projection (e.g.. 
Divide the first raw's even number by the 



Table I 
Miniature Language 



Artkh' 
a 

an 
the 



Preposition 
of 
to 
bv 



Pronoun 
it 



Conjunction 
and 



or 



Verb 
divide 
subtract 
square 



Noun 
number 
mean 
row 



/ 



Adjective 
first 
second 
odd 
even 



Matrix 



1 


3 


6 


2 
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second row's odd number). Projection time 
was the measure, and this was under the 
control of the subject. He projected the 
sentence, read it, and kept reading it until 
he could perform the operation correctly. 
This measure of total exposure time is 
sensitive and precise. It can easilv be ex- 
tended to measure suprasentencc variables 
such as overall organization. 

Exj^g^iment I. The first experiment stud- 
ied the complexity of the language and 
the complexity of the universe. This seems 
a reasonable- place to begin since un(J#^r- 
standing is concerned with the relation 
between language and events in the uni- 
verse. Two levels of ^complexity of the 
universe (two-by-two matrices and three- 
by-three matrices) were combined in a 
factorial design with two levels of complex- 
ity for the language. Complexity of the 
language was increased by adding content 
words {multiply, ratio, column, large, 
small). The difference between matrices 
was not significant, but the sentences writ- 
ten in the complex language took far 
longer to understand (6.9 seconds to 3.1 
seconds for twelve-word sentences) and 
the difference bt-twecn the two languages 
increased as sentences became longer. 

Experiment 11. The second experiment 
studied the effect of content word ratio. 
Sentences containing 50 percent content 
words (Subtract two from the mean of 
the rows) were compared to one that had 
75 percent content words (Subtract two 
from the first positive odd score). For 
twelve-word sentences, the subjects re- 
quired over iwiev as much .exposure time 
to understand the 75 percent sentences 
(10.6 seconds) as they rctjuired to under- 
stand the 50 jx^rcent sentences (5.0 sec- 
onds). 

Experiment IIL A third experiment stud- 
ied the effect of clause length. Sentences 
were written that contained the same 
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number of words but varied in clause 
lenjjth. they contained either one or three 
clauses (e.g.. Square the odd number in 
the first row. Compute the mean, square 
it, and add two.). All thirteen subjects 
understood the one clause sentence in a 
shorter mean exposure time (4.2 seconds 
to 6.3). 

This measure of understanding is an 
economical and .sensitive one. With even 
small numbers of subjects, it yields smooth 
cur\'es. Clearly, it needs to be extended to 
tasks other than mathematical ones, such 
as motor skills to learn, pictures to draw, 
objects to construct, and so on. It shows 
some promise of becoming an economical 
tool for studying the constraints between 
sentences and paragraphs. Almost all the 
current descriptors of readability are with- 
in-sentence descriptors, but surely the larg- 
est determiners of readability are con- 
cerned with overall organization, with 
constramt between paragraphs and larger 
units. 

E. Information Gain. 

The above measures are mostly con- 
cerned with the gross amount a reader 
knows after he reads a passage. But, as can 
be measured by cloze tests, the reader al- 
ways knows something about a passage even 
before he reads it. To get a measure of 
information gain, we would subtriict this 
knowledge from the knowledge the reader 
has about the passage after he reads it. 

A little reflection will suggest many uses 
for such a measure of information gain. 
In constructing materials for programmed 
instruction, for example, one is less con- 
cerned with ease of reading than with 
the efficiency with which the material 
transmits information per unit of time. 

Miller and Coleman (5) have suggested 
one way to obtain a measure of information 
gain. Tliey required their subjects to guess 
successively every word in a passage. After 
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subject iiitulr his puss In 
tin* corrvct word »is hv ^mKv^%\v^\ through 
the passagi'. Thus hi saw ouK words 
pn ifihnu ihi tniv \iv was truug to gui ss 
iiiul nut those- lollouiug it, \\v was ahh' 
to use uuiKiti ral anistraiut oulv. This, of 
ctMirsi . is a shght \ariati()u of Shannon's 
.6^ will known *'i»ntssing ganu* tccli 
nnjuc .** ll tan i;i\(' a nu asun- of what tin* 
snb|i'ct kiK)W's about the passage bi'fort' 
hr begins reading it ixprrssi-d in tirms 
of bits, but Millrr and Coleman wire ton 
lent to describe it in tenns of pertiutage 
t)l words lorreitU guessed. After the sub 
ml has guessrd e\cr\ word in the passage, 
he has terttinilv read it carefully. lie was 
then senl I h rough the passage again in 
the Aanu f.ishion. sutirssi\ely guessuig 
each word and being given the correct 
our as soon as he njade his guess. The 
dilFerente between the proportion of words 
guessed torrectl) the first time tind the 
proportion gm^sed correitlv the second 
tunc is one measure of mformation gam. 
Miller and Coleman tailed it the IG score, 
and correlated it with a number of other 
descriptors. 

The measure of infonnation gain i,s an 
important ou«' and merits devoting con- 
sideralde time and expense to its refine- 
fip*nt \niong otlu r issiu s. it pnuts up 
the distinition lu tween inherent and super- 
fhuHis e tMuplr xitv Wlnii we tr\ to imprtne 
nMdabilit\. we must distinguish between 
these two forms of iomplexit\. What we 
niainlv want to eliminate is superfluous 
(oniplexity Inherent complexity dejx-nds 
largt'ly u{x>n a passage's cargo of new" 
infomiatuMi, and we must be \er\ selective 
in n chuing that. Pee)ple an suspicions of 
readability research betaiise they sense 
that it is (Msy to improve "readabilitv'* In 
inserting unim[>ortant fillers or by reducing 
tin* leduiical prctision of the vocabular)\ 
t . bv w atering down the content. 

Paraphra^^ing tin argument. When we. 
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improve readability, we want to re<liicc 
supirfluoiis t^Hiiplexity and simultaneously 
fix the amount of new information (inher- 
ent complexity) at an optimal level. Ac- 
cording to Miller and Colenwns measure 
of IG, there appears to be an optimum 
level of redundancy. For college students, 
passages written at the difficulty level of 
fifth-grade readiTs transmitted a maximum 
of IG, 

Education necxl.s to know bow to predict 
the IG a reader will gain from a passage, 
and how to write passages so that IG is 
maximized. Of course, many people have 
expressed this idea before-usually in carp- 
ing criticisms of readability research— but 
always in a vague form that could not lead 
to tjuantifiablc readability research. 

Summary. 

One of the most important tiisks in 
mounting an experimental attack on read- 
ability Ls to develop a systematic array of 
response measures. In particular, we must 
develop efficient, economical measures of 
understanding and of information gain. 
Surely the major detenniners of readability 
lie in the constraints between paragraphs 
and higher units. However, because our 
language Ls so rich, beciiuse it ofl"ers so. 
many meeliaiiisins for recoding and para- 
phrasing and fansfomiing what we read, 
there appe^u-s to be no practical way to 
study these detenniners of readability until 
we de\vh)p tui efficient measure of under- 
standing. 

3, Relevant Populations 

Pro\iding tlie matrix of S-R functions 
needed to found a technology of readability 
is a massive undertaking. An economical 
way to begin would be by replicating 
previous studies of verbal learning, 

Tlie practical application of many of 
tln^e studies is disguised because their 
independent \ariables are described as hy- 
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polhelical tt>nslnids. Oiur ihv lennino- 
logical underhnish is cleari'd away, how- 
ever, thousands of these experiments conld 
conlrihule to our knowledge of readability 
if Ihev were only replicated on relevant 
populations-relevant populations of read- 
ers and also relevant populations of lau- 
quage ruaterials. For example, most of die 
exix»rinients of legibility and typographical 
arrangement have been made using college 
sophomores as subjects, but tht^e variables 
are not {Wtent .sources of varianc^c for adult 
readers. They lue, however, ix>tent sources 
for first graders. Perhaps many of the .e,x- 
ix'rim(»nts of legibility and related variables 
should be replicated using first graders as 
.subjects. 

llie shortcoming that reducers the use- 
fulness of most studies of verbal behavior, 
however. Is an irrelevent sample of lan- 
fpiage material. The majority of these 
studies were performed using samples of 
language materials such as nonsense syl- 
lables or disconnected lists of words. A 
great many of these studies would give 
us useful insights into readability if they 
were replicated using relevant samples of 
language materials. 

Cnirndtziiv^ to thr I.uniimi'^r Popttht- 
turn In experiiueiilal studies of readabiliU. 
thf exprriUMiilal variable is a eliara( leristic 
1)1 laiimiaue. it is a stiuiiilns diuuMisioii 
such as \\nn\ Ireqiieiuv or clause length 
The experiiiKMiter will use only a small 
sample ol wnnh or elauses-perliaps thirty 
or fortv-biil aluu)sl uie\itabl\ lie will 
elaiiu that his experimental results hold 
true for laiigiiaue in general. He will 
finundizr beyinul his sample ol langUi)ge 
material to a population of materials so 
l(»l us say that language inatenals represent 
a ^eiJtraltzdtioti tanaldc ^lu mathematical 
statistics, the tenu is random lamhle). 

Whenever an experim.Miter publishes a 
sludv investigating a language xharacter- 
islie, he IS implying that his Conclusions 



call be geiuTali/ed to a language i)opula- 
tioii. But out of the thousands of such 
studies that have been publishtxl less than 
a do/en experia.eiitt^s iH'rforiued a stntisti- 
Ciil lest that would ix^miit gcnoralization 
to the language ix>pulatioii. The c^uchi- 
sioiis of many ot these studies should be 
re-examined statistically. The laws of prob- 
ability guarantee that a certain proportion 
of them could not be replicated using a 
different sample of language materials, 

Tlie reader should remind himself that 
a significance test promises that another 
expe^menter could probably replicate the 
rCsiiUs using a different sample of the 
generalization variable. Psydiology is a 
science that studies the behavior of or- 
ganisms, so almost inevitably the geiieral- 
i/ation variable is represented by the 
sample of subjects. Tliat is, in psychological 
experiments, we want to generalize to be- 
havior, we want to be able to say that 
the experimental results could \yc replicated 
if someone else rei>eated the exi^j'riment 
using a different sample of animals. 

In experimental studii?s of readability 
and other studies of language behavior, 
other generalization variables are impor- 
tant. An experinuMilal conclusion about a 
cliaraderislie ol language is useless if it 
t annol bi' replicated using a difFerenl sam- 
ple of language i laterials. Surely no one is 
iiileresled in an experimental conclusion 
that holds true for only a dozen or so 
w (»rds or nonsense syllables. 

Ill the thousands of published studies of 
\erbal behavior (with a handful of ex- 
ceptions), however, the .statistical tests 
only guarantee that the results could be 
replicated provided the expe^-iment were 
repeated using exactly the same sample of 
language material. This is important enough 
to l)ear paraphrasing in a paragraph 
of its own. 

With less than a do/en exceptions, the 
sigliifictince tests used in most exiieriuKMils 
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(»l xctImI lu'li.iMur'clo not iHoinisc that (lit 
results coukl hv rq)lu%itrd il anollu r ( \- 
lu riiiUMiliT rciHMlccI the i xpi riim nl nsini^ 
a dilFm nl s.impic of Kiiiji^uaur malcnals. 
Tlic sitruifktiiiu' li sts promise oiil\ that the 
itHuhisions tail he ^tueriih/ed provuleil 
the ( \|H'rinieiit is repeated lusiug exactK 
the same set of languajt^e niateri»ds. 

(Consider a siaiph' experiiiieutal stiuK ol 
readahiht) that uMupares the dilFereiae 
hetweeii aeti\e verb seiiteiiees and iioiiii 
iiah/i d sentences. SnreK no one woidd he 
interisted in the conclusion of sueli iin 
experiment if he were told that it had to 
he restritleil to l\\(»nt\ or thirty particular 
Kh^lisli sentefttes. Sueh eou' lusiouh would 
he as useless as mnchisions that hold true 
t)nl\ for a specific twenty or thirty [>Ooplc. 
hi this ex[X"riment. language materials 
ha\e to represent ti generalization variable, 
llie experimenter would use only a smcdl 
sam[)le of sentemrs. hut he would crrtainU 
Wtint to generah/.e beyond them to the 
ptipulation of Engh,sh sentences. 

The purpose of tias section is to rau»e a 
tjuestion, not to pro\idi' an answer. This is 
not the time or place to go into the mathe- 
matics of statistical design. It is im[X}rtant» 
h^twt \ ( r i slablish that the re are statisli 
4 al d« siL'Ji^ th.il allow tilt experinienter to 
ij^t n« rail/* ^iutult.iiieousK acrt>ss t^en- 
tr,ih/ati»>u \,inahles. Tlie\ li»i\e been tie 
. !tl t d * !s« nhi It . si v (loK ni.tii, 1, and the 
liMM lualht in.ilkalK st»|)hisli»- att il n ft i- 
t'hi rs i^i\t'ii in that paper ). 

ft IS iitiporlanl to n<»le that the i^exienili- 
/.tfitin \ariabl» rt ptesentetl b\ lan^n.i^t 
mat* rials shoiiM b« mte n d as a lt)rmab 
\ triablt ih an\ t \\u luut nt i] ^tnd\ of read- 
alnhl\ II the (\punnt nter wi^hrs iu i^c n 
« r.th/t b(\(»nd the saiiiph t)( Kmunai^e nia 
tt riak Im' at lualK nst s in the etpirinu nt 
ti» 4 [Mipulation ol ^luh uiatt ritds. ht must 
^U^^\\ his s tnipl* uj materials \ei\ t alf- 
liilK it least as t<ut(iill\ as ht draw s his 
%anipl« oi siib|( ( ts I 't)! evaJiipl*'. in a 
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stiltb iiinipaufe4 a<.tl\( \«ib sentenies to 
iM)iiiinali/<i(itin>. t)iie \\a\ tt> tln'W a it>pre 
stiilati\t sani[)l( t>f laiglish sentences 
wt>ultl be as lt>llo\\s. eiitt r .i library^ draw 
a botik at raiidonu o[)(*n it (o ,i pai^e at 
r<uit!oiiK and read until \ini come to the 
first noniin.ih/ed sententr. Dniw <uit>ther 
book at random* open it to a page at 
random, »ind rcMd until \ou come to the 
first acti\e \erb .sentence^ and so on. Draw- 
ing a reprcsent*iti\c sample for his gen- 
erah/<itioii variable is actu<dlv more im- 
jH)rtaiit than the niathenialus of the .signifi- 
laiKv tests, howcNcr, an ex peril nentt»r 
slotbuig the charactertstics of language 
shtiuld acHpKiint himself with the ex[X*ri- 
lilt ntal thsij^n. that allow stniultaneous 
generali/ati(»n across twt) samples. . 

Perhaps, if we oinit the mathematical 
detail, it will be worthwhile to describe 
at leas< one exp<'riment»il design that per- 
mits suiitiltaneous g( iier<ili/<ition across two 
samples. The simplest experimcntid design 
that [H-nnits sinuiltaneous generalization 
simply confounds the two samples into a 
single one. Tli it is. each stibject is given a 
difTercnt set of language materials. In oiu* 
noniinah/ation-activc verb experiment, for 

example, eaili subject wt)iild read a dil- 
lereiit set of seiitences'^-iandoinly selected 
as desuibtd alune. With some (pialifica- 
tions that an* too fine to bother us. tlu* 
significance test used in an ordinarv treat- 
ments Insiibjet ts desii^n would peimit 
simtiltaiieous geiu lali/atiou across both 
samples. ' ' ' 

Jn^Josing this section, perluips we should 
nt>te cxpluitly Uiat cduCiition is concerned 
with a nuinber ot geiierali/atu>n \ariables 
t>(li( r than languai^e and subjec Is. I*'r(»- 
tpteiitK ( thtcatit)ii<il rcseauliers wish to 
gciierah/e their conchisit)ns to populations 
of te*ichers. populations of s( liools. etc. Ah 
most iiie\itabK\ tlie\ lia\( approacluxl this 
pioblem b\ leplieating inassnc. expensive 
< xpernnents a luiiiiber of times. Sometimes 
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rli»'VM lAiHTiuuMils aaild have neen pcr- 
loriPt'd at a fractioii-a small fraction such 
as one tenth or one hundredtli--of the cost 
il the gent'raii/ation variables had been 
entered aM formal variables in the experi 
mental design so that efficient designs 
'sucli as confounding the samples) could 
have bet n used in a tightly controlled lab- 
oratory study. 

Surely millioiu» of dollars ha\e been 
spent comparing the Initial Teaching Al 
phabet to traditional orthography, how 
ever, using a matched p.iirs design and a 
lightly controlled laboratory presentation 
similar to that of Staats, vt rt/. (7). the 
difference between the Initial Teaching 
Alphabet and o^iivenlional s|x*lling can be 
di monsliated with fifteen m4thed pairs of 
sul)|ec'ts. Tlie experiuK liter only newls to 
eolleet an adecjuale number of responses 
trom each viibjecl eaeli dav. Furthennore. 
the controls of the laboratory situation 
could eliminate most chances of IIa\r 
Ihorne elfects because they would reduce 
ihr role of the teacher hy programming 
or by die double l>liiid technique. As a 
second furthermore, b) confounding the 
sample of subjects with the sample of Ian 
gnage materials teadi matched pair ol 
subn'cts reads a ilillereiil. randomly drawn 
sample of lH)oks K the experiment could be 
designed to allow sinnillaiieous geiierali/a- 
liou to lht» population of language ma- 
terials as well as ihc population ol learners, 
As a third hirthermore. geiierali/atioii to 
a population ol teachers could be permitted 
by lormallv adding this to the experimental 
d(»sign as another generali/ation variable 

(Conclusions 

The major purpose of m\ two papeis 
was to advocate a program of expernnenlal 
studies of readal)ilily. To make massive 
improvements in the readtd)ihtv and teach- 
ing capacity of textbooks at Ihc adult level. 
vv(» mmi firsJ d«'V(»lop measures ot compre- 
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liensioii «md information gain. We must 
develop tcchnicjues »md instruments for 
mass producing cxixriinents at this cogni- 
tive level. 

The techniques for making such massive 
improvements in clementar> reading ma- 
terials, nowever, are almost within our 
grasp. With only a trifling investment, we 
can construct books that are so readable 
that they will completely change the notion 
of reading readiness. Almost alf the fun- 
daincijtal concepts of reading readines."?— 
the concept that shapt*s represent sounds, 
the habit of left-right progression, etc.- 
can be taught as the child actually reads 
little books. 

Let us summarize the resea/ch strategy 
that will lay the data base necessary for 
constructing such books. First we will 
analyze beginning reading material into its 
most basic units, letters and combinations 
of letters. Then we will measure the ease 
w ith which the children learn to make all 
the different reading responses to each ele- 
ment. Once the letters and letter combi- 
nations arc" thus calibrated and rank-ordered 
,as to difficulty, it will be possible to 
prepare reading materials that are far 
easier than those currently available. In 
brieL the beginning vocabulary will be 
re stricted to a few of the easier words- 
words thai iue composed ot a few of the 
easier letters. • 

Bv using cartof)iis, we H*ave prepared 
books so simple that preschoolers have 
learned to read them after a single fifteen- 
minute teaching lesson. As Figure i I 
shows, the books are not deathless litera- 
lure They are mostly pictures-only one, 
two. or Ihrec differeul words-only two or 
three tlilferent letters. These books, liow- 
ev(r. simple as they arc, arc (juilc eflective 
ill teaching two of the most basic con- 
n pts ill reading-the concepts that the 
>iliapcs re[)rcscnt sounds and the habit of 
left -right progression. 
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i Figure I 

E\tt'rptN froiu a book that most five year old*> can learn to read in a 20-nimutc lesson. In its 
last tt'st. ovt'r 75 pt ret fit of a tla^s of kitKlergarteners learned to read it after a sin>;le ksson. 
I,atei lM)uks adtl a I* u au*jt liltt rs and tnisK. tlu'v tradi llit in In lia\ing child 
words and ft adi utlu r l»asu n»nt« |)ts. that rt^d ciijoxahlr littlr books^raUier tlutn by 
\M»rds can 1m an<il\/id lulu Irlli rs and imMnon/in|L^ lonii; lists of words, letters, 
sounds that K tiers anil sounds tan In phonir lona pis. <uul spt lhng irregularities. 
r«iiilti;iaUd uilo uoiiK. and so on In A r<adahl<» hook is ah » frieieul leachiui; 
short. th» St lMM»ks U ach the concepts of mat hine. II we ttaeh th» thild the hasie 
Hading. sp< Ihni'^.. and priuliui; siiuultaiie ttuiLtpts, anil i^i\e hua hooks that he eu- 
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jov s reading, iiiutli of his pruhUiu is sdKc cI 
lie begins reading and teaching himself. 

Once we coHeel the* systematic array of 
S-U functions advocal(»d in this paper it 
will be possible^ to write little books that 
are readable by presehooli*rs after tlie sort 
of casual instruction that eould he given b) 
Captain Kangaroo on teU vision. Befcirc 
tliev <Miter kintlergarten, children could 
have read litUe cartoon books that would 
hav<* made them accomplished master^ of 
the basic concepts of die written communi- 
cation skills. 

An example* of such a cartoon book is 
given in Figure I. It is a simple matter to 
teach a four year old to read a book such 
as this. He need only distinguish three 
jibapes (I, Sam, see), and this is within 
me mental competence of a bright chim- 
panzee. 

An average five year old urn leam to 
read such a book in a single 20-minute 
lesson. Tliis book and its lesson have been 
tested in several kindergarten groups and 
have always had over 65 percent of die 
children reading atter the single lesson 

The book and the lesson are in a con 
tinuat state of refineinent. hi its last test 
(Larch Elementary School. Lennox. Cali- 
fonua). the most refined version (the les- 
son res(MTibl(»d an animated cartoon) had 
all but two of 22 children reading after 
the single lesson. In an individually ad- 
minist<-n»d test on a different book the next 
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dav, only four of the children missed more 
than half the book s 19 words. 

In short, by manipulating stimulus di- 
mensions "that affect readability, we can 
create books as readable as Figure 1. Then, 
by exploiting the child's love affairs with 
television and with animated cartoons, we 
c<m teach him to read immediately. We can 
extend Operation Head Start into Opera- 
tion Running Start— a running start for all 
our children, ^ 
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The Cloze Readability Procedure 



Eduuilors must a \alid muthud of 
finding out wlirthtT iiuslructioiul inatiruls 
an* uiidrrNlaudahk* to thtir stiuUnts, for 
studfMits must acqiiirt' uuich of tlu ir kuuwl. 
edge I)) nadiug writtru iiLstriittional ma 
ttrials. The eoneeptu»d or aestlu tie merit>s 
of a set of m»itrritd:, is ,i primary eonsider- 
ation in the srleetion of materuds. But 
materials lmi ha\e httle edueatioual \ahie 
if tliey are wTitten in Kmguage that is ,so 
complex and ohseure ehddren eaiinot uu- 
deritand tlie contents. Tlw retidtd)ility for- 
mulas presently a\aiKd)le tir** tiH) inaeeiirate 
to pro\ide educators with niueh help in this 
matter* And, while it is now possible to 
coiKstniet highl) \alid readahdity formulas 
{6\, >sueli hirniiiLis are still in the develop- 
ment stages. 

This paper will describe ways in w'hieli 
teachers uiii use tlit elo/e readability pro- 
cedure to determine if instructional ma- 
terials are understandable to children. A 
somewhat o\ rrMmplifKKl description of tlie 
elo/e readabihlv procedure includes these 
sti^ps. {d} p»Lssages are selected from the 
nuitt nal v\hose difficulty ls beuig evaluated, 
\h} every fifth word in tlie passages is 
delett\l and repKat d by underlined blanks 
of a standard Ungth, {c} the tests are 
duplicated and givtii, widiout time limits, 
to students wlio h.u< tu^t read the pas- 

Or Bornintli is an nt iiiv ^r^^l<'^•^^r n{ Kdiu alinn 
at iW liuvirMtv III (Jma^o, Ihr pn [lartttinu nt 
tlii> pap^ r uas ,u ^.tfinplisfu vl Wiih tin suppuit 
of a jjraiit frcnn ihr CvuivT Un llio Shulv oi [\u 
Kvaliiafioii (if IiislriKtiuiial Pn><uain^ it the Vni- 
MTsit) of ('aliluniiti, Los Anui Ics. 
* ritis artuir has bun n pniiti <1 {unn tlt« Aiuil 
man. issm ol UnmnUWf Ftii^ltsh, \o|iinn \I A. 
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sages from which the tests were made, (d) 
the students are instructed to write in each 
blank the word tlit^ diiiik was deleted, 
(e) res|K)iises are ^scored correct when they 
exactly match (disregarding minor mis- 
spellings) the words deleted. When the 
tests have been made properly, a student's 
score can be iiitcq)reted as a measure of 
how well he understands the materials from 
which the tests were made. 

The first section of this paper will discuss 
the validity of the clo/e readability pro- 
cedure. The sect)nd .section will describe 
how the clo/e readability procedure may 
be used to place a child in materials that 
tue graded in difficulty, and to select ma- 
terials that are understandable for a group. 

Validity of the 
Cloze Readability Procedure 

Lo^iiad or Pace Validity 

Si77ulanty to Conventional Tests: At 
first glance a clo/e Veadability test ap|>em's 
to be a complettily different kind of test 
and sonic authors liav^. made nnich of this 
fact, attempting to consbuct all sorts of 
mystical tlicoHcs about cloze tests. Some 
have professed to see a similarity between 
the processes involved in responding to a 
clo/e lest and the clozure phenomenon 
observed in the perception of geometi'ic 
figures. Indeed, it was from just tliis kind 
of conjecture* that clo/e procedtire got its 
name. 

On closer inspc»ction it can be seen tliat 
many of the items « in cloze readability 
tests identical to diose found in reading 
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camprchcnsion ivbis inddv b) uiiuniliODal 
methods and that the proccsM\s rtnjuirt'd to 
fill dozf blanks arc probably not diflcTcnt 
from those ri*(jinri*d to answer convention- 
ally rnadi* ilrnis. 

Conventional completion test items are 
nuidc by simply Jektint; »i word or phrase 
from a sentence. For example, given the 
sentence The hoij.s rode liorw^, it i^ pos- 
sibli* to make the cx>mpKtion <jucstions,- - 
• -rvdc /imcy, Tlic hoifs- - - The hoys 
' - - 'lior\c\, and The boys rode- - . Tlic 
familiar vvh- (jucstions arc made in much 
the same manner. A word or phrtise is 
delt^tl, a vvli phrase is inserted in its 
placr» ami the sentence is transformed so 
that it begins with the vvh- phrase. Tlii>s 
gives the questions. Who rode liorse\? Wluit 
did tlic boys do? Mliat did Oie hoys do to 
the horses? and What did the boys ride? As 
in clo/.e tests, the correct answers to these 
(juestions are the words or phrases deleted. 

Contrasts with Conventional Tests: But 
items made by clo/e »iJid conventional 
test making procedures differ in several im- 
jx>rtant resjx'cts. First, m a clo/e reada- 
bility tcst» only one word is deleted at a 
time while in conventional tests, whole 
phnises and clauses mtiy also be deleted. 
Further, in clo/e readability tests, struc- 
tural words may be deleted. (Structund 
words consist of classes such as artickvs^ 
prepositioas, conjunctions, modal and au.\- 
ihary verbs, ete.) But in conventional tests, 
only lexical words (consisting roughly of 
verbs, nouns, adjectives, and adverbs) may 
be deleted by themselves. 

A sOGt)nd major difference is the fact 
that clo/e readability te^sts iu*e made only 
from th(» sentencx*s in tlie text while* con- 
ventional tests iteriLs mtiy be made either 
from sentences in the text or from tlic 
sentenct's that can be derived from the 
text (5). A sentence intiv be derived by 
any. or all, of three priKt\sse.s. Tlic simplest 



type of derived sentence is obtained by 
transforinii:ig the sentence. For example, 
the sentence The hoys rode tlic horses can 
be transformed into Horses \icrc ridden 
by the boys* and then transformed again 
into questions such as By whom were the 
horses ridden? Sentcnics can also be de- 
rived by substituting svnonymous words 
or phrases for the words or phrases in the 
sentence in the text. Finally, derived sen- 
tences may be obtaintxl by explicating the 
statements implied by the fact that two 
NCiitcncrs are cx>ntiguoiis. Consider the sen- 
tence, The hoys got home first followed by 
They rode liorscs. The contiguity of these 
sentences implies the sentence Riding the 
horses caused the boys to gci home first. 
Only conventional test items can be made 
from sentences derived from but not actu- 
ally in a text. 

The third major eoiitr<ist is the fact that 
clo/e readability tests »u^e t«ikcn by students 
who have not read the undeleted version 
of the passage. 

Probably too much has been made of 
these contrasts belAvecn cloze and conven- 
tional tests. The student has eighty percent 
of the te.xt on which to base his responses', 
so his responses very mudi dejxjnd on his 
ability to understand the te.xt. Also, the 
fact that be has not read the original text 
may rccpiire that he uses processes similar 
to thost reqiiired to answer questions made 
from derived sentences plus a sensitivity 
to the authors style and the tone of the 
passage. However, the contritsts do exist 
and so the (jucstion must be referred to 
the researcher, die final arbiter of such 
disputes. 

Concurrent Validity 

Tliere have been intiny studies in which 
the objectives were to determine if scores 
on clo/e readability tests correlate vvitli 
scores on conventionally made comprehen- 
sion tests and if the cloze difficulties of 
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passages correlate with the difficulties of 
the piis.sage^ ds determined by conventional 
tests. By now, tins literature has grown 
too liirge to review in detail in this paper. 
Since extensive analyses of tin's research 
are available in Rankin (13) ^nd Burmuth 
(5) only a few of the most decisive studies 
will Ix* mentioned here. In general, these 
studies seem to show that clo/e and con- 
ventional tests measure the same processes. 

Validity a Measure of Comprehension 
Abilittj. Clo/e tests seem to mei\sure a 
wide variety of comprehension responses. 
Wilson Taylor (18), the originator of the 
clo/e procedure, found a correlation of .76 
between scares on a elo/e readability test 
and scores on .i multiple choice test made 
from the same passage. Bormuth (3) con- 
structeil tests to measure the cx>mprehen- 
sion of vociibulary, explicitly stated facts, 
se(juenct*s of events, inference's, causal re- 
lationships, main idea, and author's motive 
ni eacli of nine passages, IK* gave these 
and the clo/e readability tests to eU'menta- 
ly sdiool students. The correlatioas be- 
tweeii the cloze and etJiiventional tests over 
each passage ranged from .73 to .84. Whtm 
the correlations were corrected for the un- 
reliabilities of the tests, the c /rrelatioiis 
approached 1.00. When tlie cor^ventional 
tests were stored to obtain scores for cacli 
type of item, and then factor analyzed 
along with the cloze test scores, only one 
factor emerged, Kohlcr (11) performed a 
somewhat similar factor analysis stxidy 
using tenth-grade students and obtained 
similar revults. 

To determine if cloze readability tests 
measure some of the subtler respoascis to 
language, Bormuth and Hook (9) studied 
tl;e correlations between clo/e readability 
tests and tests of ability to recogni/e an 
author's style. Two tests of ability to rec- 
ogiii/e authors' styles were made by select- 
ing short p.issages from several of the works 



by an author, works that the students in 
the experiment had not read. The passages 
were then mixed with passages from works 
by other authors and given to college stu- 
dents who had just studied two books by 
each of the authors. Scores based on ability 
to correctly identify the passages by an 
author had significant correlations with 
scores on cloze tests made from passages 
taken from works of the author. 

Validittj as a Measure of Comprehension 
DiffieuUij: The mean percent of items a 
group .inswers correctly on a cloze read- 
ability test seems to provide an accurate 
measure of the difficulty of the passage, 
almost regardless of how difficulty is meas- 
ured. Sukeyori (16) found a correlation of 
.83 behveen tlic cloze readabilities of pas- 
sages and the combined subjective ratings 
of the passages by three judges. Bormuth 
(8) found a correlation of .92 between the 
clo/e readabilities of passages and the diffi- 
tulties of tlie same passages as measured 
by multiple dioice tests. Subsequently, 
Bormuth (2) used each of the four forms 
of tlic Gray Oral Reading Paragraphs. He 
found correlations ranging from .90 to .95 
between the cloze iUid word recognition 
difficulties of the paragraphs and correla- 
tions ranging from .91 to .96 between the 
clo/e and the comprehension difficulties of 
tlie paragraphs. 

Experimental yalidittj 

One study (8) approiiched the problem 
of clo/e readability test validity experi- 
mentally. Passages were varied systemati- 
cally in language complexity and subject 
matter. Both cloze and multiple choice 
tests were made over each passage and 
given to students at tliree different grade 
levels. Each factor and the interaction be- 
tween language complexity and subject 
matter produced significant effects on both 
kinds of test scores, llie effects were ap- 
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proximately proportional on the two kinds 
of test scores. 

Validity of thc^^^ccdurc Itself 

A cloze test be miide, iidmiiiistered, 
scored, and interpreted in a number of 
ways. The particuhxr set of alternative pro- 
cedures actually used in tlie cloze readabil- 
ity procedure represent what researchers 
have found are the best procedures when 
validity, economy, and convenience are 
considered simultaneously. 

Test Comiructivii: The practice of delet- 
ing every fifth w ord is followed bec»iusc it is 
simple and economical to u^e and beuiuse 
it provides the greatest number of items 
possible for a given passage and thereby 
provides the most reliable measure of ptiss- 
age difficulty. Wliile deletions may be less 
freijuent than every fifth word. MiicCini^ie 
(12) has shown that when a deletion sys- 
tem le<ives less than four words of coiil|e\t 
between items, a student s ability to ani>)\er 
an item begins to depend heavily ujpon 
whether lie wa.s able to cUiswer corrtictly 
the adjacent items. When this tKCurii^ the 
scores become difficult to inteq)ict iti «uiy 
meaningful way. j 

Administration: Cloze tests may be ad- 
ministered either with or without die stu- 
dent reading the piissage from which the 
test was made. Researcli shows tliat tlic 
two ni *thods are about equally valid. Tay- 
lor (17) found that scores on clpze tests 
administered after students had read the 
passage had slightly higher correlations 
with scores on comprehension tests. Rank- 
ins ( 14 ) studies showed the same results. 
But the data in bodi studies showed that 
this effect probably Wtis tlie result of scores 
being somewhat more vimable than when 
the students had not read tlie passage, an 
effect that is more economiciilly and easily 
obtained by sunply adding a few items 
to a test. Tlierefore. lx*Ciiuse of the savings 
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iu testing time and preparation of materials 
and because results which are just as valid 
ciln be obtained, it is the most desirable 
procedure to give the tests to students who 
have not read the passages from which the 
tests are made. 

Scoring: A students response can differ 
from the deleted word in semantic mean- 
ing, grammatical inflection, and spelling. 
Users of the cloze readability procedure 
have settled on the practice of scoring 
correct just Uiose responses which exactly 
m»ilch the deleted word, but minor mis- 
spellings are disregarded. This practice is 
based on findings by Taylor (18) Rankin 
(14) and Ruddell (15) that including 
synonyms as correct responses slightly in- 
creases the correlations between cloze 
scores and scores on comprehension tests. 
But their data show diat it does so simply 
by increasing the variability of the scores, 
an effect that is for more easily obtained 
by adding a few items to the test. 

Bormuth (7) chussified resjx)nses into 
three uitegories depending on whether root 
foiTns of the resjwnses were identical to. 
svuonymous with, or semantically unrelated 
to the word deleted and then further 
classified the responses in each category 
according to whether they were grammati- 
cally correct or not. Only the grammatically 
correct resjwnses had signifiaint correla- 
tions with a comprehension test. When the 
scores based on ' the responses exactly 
matching the deleted words were held con- 
stant, all tlie other correlations dropped to 
zero. Hence, responses that exactly match 
the deleted words furnish tlie most valid 
ineiLsures of comprehension. 

Interpretation. Tliere is some value in 
knowing diat one passage is more difficult 
for students tlian another. But a cloze read- 
ability score has little value unless a teacher 
can say that the score does or does not 
represent a satisfactory level of perform- 
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ance on the materials from which the test 
was made. A standard has long been ac- 
cepted (19 J for conventional coniprehen- 
5ion tests and this standard is widely^^used 
in practice (1.10). It asserts that materials 
are suitable for use in a child's instruction 
when he is able to answer correctly 75 
percent of the questions asked him about 
the materials. The materials ar^ said to be 
suitable for his independent study when 
he can answer 90 percent of the items. 
Bonnuth (2.4) found that a score of 75 
percent on conventional comprehension 
tests is comparable to a score of 44 percent 
on a cloze rwidability test made from the 
siime passage and that answering 57 per- 
cent of the cloze items is comparable to 
answering 90 percent of the items on c*on- 
ventional comprehension tests. 

Summary 

In general, the studies of the validity of 
tests made by the cloze procedure seem 
to justify four assertions. First, cloze read- 
ability tests provide a valid measure of a 
students reading comprehension ability. 
Second, tlie cloze readability procedure 
provides a valid method of measuring the 
comprclioiisiou difficulties of passages. 
Third, tlic procedure itself seeins to in- 
corporate both the most valid and the most 
economical of the jx>ssible alternatives for 
designing a clo/.e readability procedure. 
Finally, cloze readability scores can be 
usi\! to judge the suitability of materials. 

Applications 

Perhtips the most im{X)rtant advantage 
of the cloze readability procedure is that 
it recjuirt^ little training in testing tech- 
nology of the i>erson who wishes to use 
.the procedure in many of its imix>rtant 
applications in schools. This section will 
describe two of the most im[x>rtant types 
of applications. 



Piecing Students in Graded Materials 

One of the most critical tasks a teacher 
must perform in .reading instruction is to 
place the students in basal or other practice 
reading materials that have the appropriate 
level of difficulty. The materialS'must pre- 
sent enough difficulty to permit practice 
of reading skills but not so much difficulty 
that the student develops faulty reading 
habits and feelings of frustration. The 
method of testing materials that is presently 
advocated calls for the teacher to have the 
student read orally a sample passage from 
each level pf materials and then answer 
questions about what he just read.^ The 
teacher notes the adequacy of both the 
child's word recognition and comprehen- 
sion. This method has undeniable advan- 
tages, but because it requires much time to 
administer and a relatively high degree of 
training on the part of the teacher, it is 
seldom used. 

The cloze readability procedure can be 
used to make a set of tests which permit 
the rapid and accurate assignment of stu- 
dents to materials having an appropriate 
level of difficulty. The basic steps of this 
procedure consist of making a cloze test 
over each of the texts in the series, ad- 
ministering them to the students, scoring 
the tests, and placing the students in the 
materials on which their cloze readability 
score fell into the appropriate range. 

The most critical part of this procedure 
is to select a test that best represents the 
difficulty of a text. This is done by initially 
making several tests from passages random- 
ly selected from the book, finding the mean 
difficulty of the entire set of tests, and 
then discarding all but the test having a 
difficulty closest to the mean difficidty of 
all the tests. 

A step by step description of this process 
includes these steps, (a) Select several pas- 
sages from a book, say from six to twelve, 
using ,ome random method for selecting 



LC 



49 



Tin: Our/i; Hi'.\n\mi,ri y Piuk kuuuiv 



45 



the passages. Each passage should begin 
at the normal beginning of a paragraph 
and each should be at le;ist 250 words in 
length. Tlie test, for reasons of both con- 
venience and test reliability, should contain 
exactly 50 items, (b) The tests are then 
given to students at the grade level at 
which the text is moc»t commonly used, 
25 to 30 students iire usually enough to 
obtain reliable results, (c) The mean scwe 
on each test is calculated and then the mean 
of the mean scores Ls calculated, (d) 
Finally, the test whose mean score is closest 
to the mean of the entire set of tests is 
selected and the rest of ^the tests are dis- 
carded, (e) When a test has been selected 
for each of the texts a teacher is likely to 
use, the tests can bt mimeographed and 
compiled into booklets which can be ad- 
ministered iis group tests. When a student's 
score falls betwtvn '14 and 57 jxirtvnt on 
one of the.se tests» the materials arc at the 
level of difficulty thought to be suitable 
for use in his suiKT\Lsed instruction. 
Materials on which a student's st*orc is 
above 57 jKrcent are suitable for use in 
his independent study. 

Tlie reliability of this prcxedure clejxinds 
upon four factors. First, if longer tests are 
used, the students' scores will be more 
accurate, but the tests will also rcijuire 
more time to administer. Secoinl, if ti larger 
number of tests arc made when selecting 
the test to represent the materials, the tc^st 
selectee! will more accurately refle^ct the 
difficulty of die materials. If the te5»ts are 
to be used frecjuently, it is probably worth- 
while to sele*ct the tc^st from a hu-ger num- 
ber of tests. Third, some materials vary 
unevenly in difficulty as they proceed. These 
materials are pcxjrly cx)nstructed and tlieir 
use should, if possible, be avoided. Fourth, 
the procedures outlined for c*oastructing, 
administering, and scoring tlie cloze tests 
mast be foUowcxl exactly or the results 
will be meaningless. 



Selecting Materials for a Gti^up 

Wlien materials are being c^asidered for 
adoption by a school, one of th^ important 
cjuestions that must be considerecl is wheth- 
er the materials are understandable to stu- 
dents. While the c]uality of the conceptual 
content of materials is of panmiount 
importance, that content has little likeli- 
hocxl of being learned unless it is present 
in an understandable manner. The cloze 
readability procedure can be used for eval- \ 
uating the suitability of materials for a 
group. 

Criteria for Evahuition', When the same 
text is to be used with all the students in 
a group, two CTiteria must be used in 
judging the suitability of the text. First, 
the materials whose difficulty is most ap- 
propriate lu^e those on which the largest 
iiuiubrr of students am demonstrate a 
.satisfactory level of performance. Second, 
the best materials arc tliose in which the 
level of difficulty is fairly uniform through- 
out the materials. Eith(?r all sections of the 
book should exhibit about the same level 
of difficulty or the sections should gradually 
bec*ome more difficult iis the student pro- 
ceeds through the book. 

Designing the Tests: From the point of 
view of accuracy in evaluating materials, 
it would be ideal if a clcr/e readability 
test were made over the entire bcx)k. But 
practical necessity demands that testing 
time and materials prepiiration be kept at 
a minimum. Conse^cjuently. a proccxlure for 
sampling passages is usually refjuired. If 
done carefully, sampling technicjues will 
lead to accurate results, 

Tlie best plan is to divide the book into 
sections, select two or more sample pass- 
ages from each section, and use those p»iss- 
ages to make the cloze tests. The passages 
should be selec*ted by some random 
process. For example, the paragraphs in a 
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section inij^lu hv nuinlK»rrcK the- iuiiiil>tts 
written on slips of paper, the slips sliutHal 
and two or more slips drawn with the 
restriction tliat no two passages should 
contain tlie same material. 'Die passages 
may be as short as a paragraph or as long 
»is seems to make a convenient test. (A 
250 to 300 word test fits cicely on one 
sheet of paix»r.) But each sample should 
be a Icngtii of continuous text, and it should 
begin at the beginning of a paragraph. 
The samples need not be of exactly identi- 
cal length since the cvaluator will be work- 
ing with percentage scores. 

Sclcctinp, the Students: The students to 
whom the tests .u*e gi\en should be eitlier 
the entire {>opulation who will use the 
brK)k or a sample of students who arc 
truly representative of that population. A 
good sample of students can be obtiined 
in" randomly drawing one or more students 
from ever\' cLiisroom in which the book 
will be used or by numbering the students 
in the population and randomly drawing 
die re(|Uired number of students, 

Adminhlerin^ the Tests: For practical 
reasons, it is seldom desirable to give all 
the tests to ever\' student. It is perfectly 
penntssible to break the group up into 
subgroups and administer a fraction of the 
tests to each subgroup. But the selection 
of subgroups should be done by a random 
inetluxl to avoid biasing Uie means of the 
indiviiliial tests. 

Aualtjzmg the Restdtst Tlie analysis of 
studenU' scores iK'gnis by converting each 
score on each passage* separately into a 
lx»rciMitage score. A itseful analysis is to 
detcnmne what proiM>rtion of the scores 
were 44 percent or larger. This provides 
a ine*Lsnre of the projwtion of students 
fnmi whom the l>ook, as a whoU\ Is or Is 
not suitable. Another analysis is to cidculate 
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this prO{X)rtion of scores but to c*dcnlate it 
septU-ately for each [Xissage and for each 
section. When a frcnjuency distribution is 
made of these proix)rtioiis or when their 
standard deviation is calculated, tlie evalu- 
ator obtains a measiu-e of how variable 
the different sections of the book are in 
difficulty. The evaluator ain also plot a 
graph of the section difficulties to deter- 
mine how the difficult)' clianges across 
sections, but for this purj^ose it is best to 
calculate the means and plot them. 

Comparing Alternative Texts: If the 
evaluator has some training in statistics 
or has access to a consultant who is compe- 
tent in these matters, it is possible to com- 
pare texts and even compare two or more 
textbook series. Using an analysis of vari- 
ance and a carefully designed procedure 
for selecting the sample passages from each 
text or series it is ix)ssible to find out if 
texts difler in difficult)', if the texts differ 
in the variability of their difficulty, and if 
die texts increase systematically in difficulty 
as they proceed from section to section and 
text to text. 

Assigning a Grade Placement to a Text: 
Often it is not possible to administer the 
cloze readability test to a representative 
sample of the students who will use it 
And even if it is ix)ssible to do so within 
one school system, the results are seldom 
geiierali/able to other scliool systems. Con- 
se(iiiently, it may* be desirable to express 
the texts difficulty in a more generally 
useful form, as a grade placement. When 
a grade placement number is attached to 
a book, it can be interiKcted as the average 
level of reading achievement attained by 
cJiildren who are able to read the book at 
the minimum level of comprehension. 

The method is fairly simple, but it re- 
quires having a re;lding achievement grade 
placement score oh each student and it 
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ri'quinvs some amount of calculalioii. The 
gradi' plactMniMit score Is first com»hil'»cl 
with the stucltMits clo/e readability per- 
centage score. Then, using the t*i)innion 
regression prediction fonnuhi. the evahiator 
calcuhites the reading grade placement 
score that is comparable to a score of 44 
jXTceiit on the cloze readability test. This 
is the number assigned as the grade place- 
ment of the material. It c-aii be calculated 
either for each passage in the text or for 
the text as a whole. The grade placement 
number is then useful to teachers in other 
schools, who, if they use the same readiuK 
achievement test, can calculate the pro- 
lK>rtion of their students who will be able 
to understand the text at a minimum level 
of coinivtence by counting the number of 
stuch'iits who ha\e reading achievement 
scores falling above the grade vl.itement 
number assigned io the text. 

Concluding Remarks 
r.ducati»rs ha\e long felt the need to 
adapt instruction to the individual differ- 
ences among their students but their efforts 
to do so have often been hamjX'red by the 
absencx' of practical procvdnres for attain- 
ing this objccti\e. An important way to 
adjust instruction is to plice in each child s 
hands materials having a level of difficulty 
tliat is appropriate to liLs le\el of reading 
skill. Tlie clo/e reiulabilitj- procedure pro- 
vides S(»veral de\ia\s to help acvomplLsh 
this <)bjecti\ e. 
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